Predictive Analytics – the precog, the home run and the future of analytics
Many moons ago at a lovely big corp near you, which had the proverbial big internal tech project to improve and simplify things.
And it was a feeding frenzy
Guess a few big consultancy names and they were probably involved.
So now you’ve visualised the budget lets call this the Monster Platform Waterfall Project, or MPWP for short. And of course the management loved it, but not as much as the external consultancies.
So you’ve already guessed that it was a death march. And everyone operationally involved with experience knew it. But it still kept on going.
Which was a shame
Because all this could have been foreseen using predictive analytics with a strong degree of accuracy. And it all started by a young man called Thomas Bayes from a small 18 century town in Hertfordshire, ®U.K.
This is probably Bayes Theorem…
Bayes was an 18 century English statistician and philosopher and absolute legend in predictive analytics who came up with the mathematical theory called Bayes’ Theorem which was eventually presented to the Royal Society under the name “An Essay towards solving a Problem in the Doctrine of Chances”
Hugely simplified, Bayes’s theory of probable outcome, when operationally applied, eventually became a central tenant behind the theory of risk.
And so this is a short story of how a young man from Hertfordshire became the man whose theorem now runs the modern world and what it means for the future of business analytics.
They can see the future, sort of…
Moving forward a couple of centuries from Bayes’s life we now have the fascinating tale of the birth of the modern forecasting industry and the fates of opposing economists John Maynard Keynes, Irving Fisher and Roger Babson, as told in the book by William Sheldon and covered in this excellent article in the Financial Times by Tim Harford, from which I will now summarise.
Fisher built a business specialising in forecasting but failed to predict, like Keynes and many others, the Great Wall St crash. But unlike, Keynes, failed to recover his finances when the madness of the panicked crowd bankrupted him as he leveraged up loss making positions.
Keynes did this because he realised long term forecasting was unreliable, but learnt to focus in on well run companies who pay a dividend.
Meanwhile Babson, although not right on everything, got rich on his forecasting.
The Wall St crash was not a disaster, for him it was a calling card.
Zoom forward to 1987 and we reach the point when a psychologist called Philip Tetlock decided to scientifically study how well experts could do predicting future geo-political events….and then waited 18 years for the results to come in.
And the conclusion was, experts across all disciplines failed to see the future.
Which was a bummer
Tetlock, not a man for rash sub 18 year judgments, or to be put off in general, was undeterred and in 2011, along with others including the support of a U.S intel agency, launched the Good Judgment Project, which allowed people to log on and make anonymous predictions on subjects they believed they had competence in and check in occasionally to see how the events were going.
And you’d expect them to be wrong again.
But they weren’t
Major events were being predicted much more accurately than expected. And further more, “super forecasters” were identified whose predictions were considerably beating the odds. In fact, “the super forecasters were able to sustain and even improve on their performance”.
So within the wisdom of crowds there’s an inner circle of out-performers.
Or, if you’re a fan of the film Minority Report, “Precogs” because they apparently require a degree of anonymity, autonomy and conditioning to perform optimally – which is scarily like the movie…
And Precogs, it seems, can be built
Well, at least improved, continually…
Tetlock identified that applying some discipline the super forecasters predictions could be optimised, which he summarised under the acronym CHAMP, which are broadly as follows;
- Comparisons are important
- Historical Trends help
- Average opinions
- Mathematical modelling is useful
- Prediction bias exists and can be addressed
According to Tetlock, super forecasters are also intellectually “foxy” in nature, thinking broadly, intuitively, self critical and ad-hoc.
And the smart money is that he’s right. But now let’s get back to how this links to business analytics.
So lets talk baseball…
“You’re not solving the problem. You’re not even looking at the problem”
Billy Beane, Money Ball
During all this there was a growing upheaval in Baseball performance analytics largely started by Earnshaw Cook, who James Surowiecki in his article in the New Yorker referred to as the “Buffett of Baseball”.
Cook’s work would later evolve intro a analytical approach called Sabermetrics in honour of the Society for American Baseball Research by a guy called Bill James. Later on the operational application “baton” well and truly proverbially picked up when an ex-baseball player turned Oakland A’s manager called Billy Beane believed he could win the Majors by looking at Baseball differently.
Although he didn’t call his new analytical approach by this name, it fell broadly under the sabermetrics principals and an already growing industry got a massive steroid injection (little baseball joke there…) as covered in Michael Lewis’s book Moneyball and the subsequent eponymous film with Brad Pitt.
If I could sum up the Sabermetrics methodology while quoting from the film it would be;
Do they get on base?
You see, Billy Beane’s Oakland A’s dramatically and consistently out-performed the big wallets of baseball by looking at player stats differently, including hyper short term outcomes that had a link with winning games.
In business terms it would be to leave the MBA, jovial patter and big club heritage at the hiring room door.
What mattered was the key result outcomes (“Get on Base”) that got you to the Objective (“Win more Games”).
So where does that leave me as the modern CEO or founder that I obviously am?
While Tetlock’s original experts had to predict the outcome of multi-faceted geo-political scenarios (think Ukraine, Libya or even Scotland), the business owner or lead has just to predict relatively short term reasonably well defined outcomes.
Or how often does my team get on base
Now there are many ways to skin a cat and in my recent survey on startups in Tech City area of London which showed a mass 40% migration of all startups to the Objectives and Key Results (OKR) goal setting methodology used by Google, Linkedin and many other household names.
OKR produces a well-defined and reduced bias grading method for setting and reviewing goal outcomes and thus a framework for measurable, comparable, historical analysis of short-term goal setting success.
So the “O”, “K” and “R” as a measurable framework, is a direct match of the “C”, “H” and “A”, and the “M” plus “P” of Tetlock’s observations on superforecaster optimisation.
And if you ever watch the Hollywood version of events in the film Moneyball, the upbeat background Act I > II music kicks in when Billy starts sharing his methodology and motivating his team. Well, the first part of that is the core fundamental of OKR in a baseball glove-shell.
But without the brutal trading, if you lose, obviously…
So if you want to predict if you’re going to achieve that business home run?
It’s looking pretty probable (sorry…) that if you break your big hairy objectives down into short measurable outcomes using OKR and then ask the huddle of project managers, coders, C-levels on the pitch or watching closer from the stands to anonymously forecast what the outcomes will be, you’re going to start seeing the near-future much more accurately and pretty soon you’ll also find your Precogs.
Because the view of these guys are also the barometers for the macro economical hurricanes and market slipstreams that your particular corporate vessel sails through – and the clever ones will learn what can and cannot be predicted against the wider storm.
And that is a big deal for your business
And if the Tetlocks and other pioneers in predictive analytics keep making progress, it’s an even bigger thing for the world in general.
Now if all this was in place, would it have predicted the outcome of MPWP?
Because the Monster Platform Waterfall Project was a 3 year+ play and had a massive scrum involved there would have been a sizable enough data there to forecast future milestone outcomes and some “Project level” Precogs would surely have emerged from the mist.
So yes on that front, but probably no until early-mid project stage
MPWP would have already gone through several phases (bases?) of exploration, specing, management sign offs, tendering and back room corporate horse-trading before it hit the operational high road and the checked shirts got involved. And probably no one goes to be the screaming canary in the mine in the first couple or so kick off meetings.
So in balance, like shooting down a charging bull, it would have died a faster death and some of it’s more outrageous components (there were parts with logistical value) could have been rescoped earlier.
But let’s not underplay the value of that – because for projects on this size that would have produced cost saving in the tens of millions.
And more importantly, we’d all have hit fewer but more meaningful bases, worked better as an organisation and delivered something of higher value.
Which is what the Grand Fromages and us on the ground all ultimately wanted to do.
So what would Bayes think about all this?
For that we know very little
In fact, we don’t even know if Bayes would have endorsed his own theorem. You see, Bayes died before publishing it.
Bayes Theorem was published post-humously by a man called Richard Price who collected Bayes’ original doctrine and notes, edited them and published them ultimately leading them to be presented before the Royal Society in 1763.
And so, in what is surely the biggest probability irony of all times, the man whom theory conquered the modern world failed to see it, yet alone predict it.
Now who could have predicted that?
For any of the curve balls I’ve clumsily tried to combine and pitch to you here I am standing on a very tall stump created by the following;
Michael Lewis, Moneyball
Tim Harford How to See the Future
William A Sherden, The Fortune Sellers
Philip E Tetlock Expert Political Judgement
The Good Judgement Project Good Judgement Project
Earnshaw Cook, Baseball Analytics
The New Yorker The Buffett of Baseball
Bill James Baseball Extract
Thank you for taking the time to read this post. If you know people who would be interested, please feel free to share.
Images: Minority Report; Property of Twentieth Century Fox Film Corporation, DreamWorks SKG, Cruise/Wagner Productions. Moneyball; Sony Pictures [United States]. Fortune Teller painting: Women’s Work Telling Fortunes, Harry Herman Roseland