The Folly of Prediction (World Cup Style)

Anybody else having a problem with these seemingly scientific “odds of winning” statistics being thrown about by Bloomberg and Five Thirty Eight around the World Cup? Here’s the chance of England beating Uruguay today from Bloomberg:

Bloomberg World Cup Predictions

And Nate Silver’s new FiveThirtyEight Blog’s advanced look:

Five Thirty Eight Predictions

 

Now, we’re big fans of Nate Silver, and wouldn’t mind seeing more data in journalism as a whole – but this whole exercise of providing odds of winning seems a bit off to us.  For one, it’s been a little annoying to hear people around town saying things like Australia has a 9% chance of winning versus the Netherlands, like they just measured the temperature outside or are quoting a stock price.Moreover, there have been some really off “predictions”, like Spain having a 53% chance to beat the Netherlands in its opening game, but losing 5 to 1 and being eliminated from the whole tournament in just two games. For you non soccer fans, that’s like the Seattle Seahawks making it back to the playoffs next year and losing 56 – 3.

The temperature, your height, and IBM’s stock price are all measurable things, with actual scientific answers. The percentage probability that England will beat Uruguay is not measurable, and there is no correct value for it. Despite the pretty graphics and official sounding “predictions”, there’s actually no statistic you can assign to such a prediction. There’s just three possible outcomes – a win by Team A, a win by Team B, or a tie. So technically, the odds of winning are 1 out of the three possibilities, or 33%.   But the data journalists take it much further – with Nate Silver explaining that they use the SPI algorithm (Soccer Power Index) to come to such predictions, which is in fact quite thorough, looking at each players stats and performance between international and national play. (more on SPI see here and here).

Mr. Silver earned the right to be making soccer predictions by doing a great job predicting elections.  But predicting elections is a bit different, as there is statistical data of registered voters, election turnout data based on weather, population sizes of counties, and people who actually get asked who they will vote for. You get “inside information,” in a way. You get to know how the players are going to play the game before the game is played. Nate Silver’s genius was stripping out all the biases and noise in those polls and zeroing in on what the right ‘inside information’ was.

But analyzing the past performance of a soccer team and its players isn’t “inside information,” it’s past performance. And past performance has a nasty way of being a terrible indicator of future performance, especially when you get 11 different people on each team. At the very least, that’s 22 degrees of freedom put into the mix. Add in what they ate the day before, how their feeling, whether  they got in a fight with their significant other, the weather, the referee, and all of the rest; you’re talking hundreds if not thousands of moving parts. That is – as they say – why they play the game.

Why do we care?  Because there’s a whole lot of prediction nonsense in the financial markets that is almost continuously wrong:

–          Stock price targets

–          End of year Dow or S&P targets

–          GDP targets

–          Unemployment

–          Housing Starts

–          Crop Reports

–          And the list goes on and on…

Where is Nassim Taleb on data journalism and spurious correlations and the role of luck and all the rest? Where is Barry Ritholtz on the folly of prediction (see here, here, and here)?

To be fair, Nate Silver does point out the more complex algorithms does mean there is more possibilities for errors, and he has done studies “checking their work”. Still, the world cup predictions are just that, predictions. They are saying more that this team will beat that team about 65% of the time, all else being equal and if they played 10,000 times – than they have a 65% chance of beating the other team today! It’s semantics, but we think it’s important to bear in mind.

The World Cup is one thing, but what happens when the data journalists start posting likelihoods of the market finishing the year up such and such, and real people with real money act on it. As Freakonmics put it in a podcast a few years ago:

It’s impossible to predict the future, but humans can’t help themselves. From the economy to the presidency to the Super Bowl, educated and intelligent people promise insight and repeatedly fail by wide margins.

Maybe the data journalists should be regulated and have to put big ‘Past Performance does not necessarily guarantee future results’ disclaimers on their articles, or at least admit that while fun to do, and good for page views, the odds of correctly assigning all of these probabilities is about the same as the 1 in 9.2 quintillion chances of perfectly filling out the March Madness bracket.

 

Write a Comment

Disclaimer
The performance data displayed herein is compiled from various sources, including BarclayHedge, and reports directly from the advisors. These performance figures should not be relied on independent of the individual advisor's disclosure document, which has important information regarding the method of calculation used, whether or not the performance includes proprietary results, and other important footnotes on the advisor's track record.

Benchmark index performance is for the constituents of that index only, and does not represent the entire universe of possible investments within that asset class. And further, that there can be limitations and biases to indices such as survivorship, self reporting, and instant history.

Managed futures accounts can subject to substantial charges for management and advisory fees. The numbers within this website include all such fees, but it may be necessary for those accounts that are subject to these charges to make substantial trading profits in the future to avoid depletion or exhaustion of their assets.

Investors interested in investing with a managed futures program (excepting those programs which are offered exclusively to qualified eligible persons as that term is defined by CFTC regulation 4.7) will be required to receive and sign off on a disclosure document in compliance with certain CFT rules The disclosure documents contains a complete description of the principal risk factors and each fee to be charged to your account by the CTA, as well as the composite performance of accounts under the CTA's management over at least the most recent five years. Investor interested in investing in any of the programs on this website are urged to carefully read these disclosure documents, including, but not limited to the performance information, before investing in any such programs.

Those investors who are qualified eligible persons as that term is defined by CFTC regulation 4.7 and interested in investing in a program exempt from having to provide a disclosure document and considered by the regulations to be sophisticated enough to understand the risks and be able to interpret the accuracy and completeness of any performance information on their own.

RCM receives a portion of the commodity brokerage commissions you pay in connection with your futures trading and/or a portion of the interest income (if any) earned on an account's assets. The listed manager may also pay RCM a portion of the fees they receive from accounts introduced to them by RCM.

See the full terms of use and risk disclaimer here.

Disclaimer
The performance data displayed herein is compiled from various sources, including BarclayHedge, and reports directly from the advisors. These performance figures should not be relied on independent of the individual advisor's disclosure document, which has important information regarding the method of calculation used, whether or not the performance includes proprietary results, and other important footnotes on the advisor's track record.

Benchmark index performance is for the constituents of that index only, and does not represent the entire universe of possible investments within that asset class. And further, that there can be limitations and biases to indices such as survivorship, self reporting, and instant history.

Managed futures accounts can subject to substantial charges for management and advisory fees. The numbers within this website include all such fees, but it may be necessary for those accounts that are subject to these charges to make substantial trading profits in the future to avoid depletion or exhaustion of their assets.

Investors interested in investing with a managed futures program (excepting those programs which are offered exclusively to qualified eligible persons as that term is defined by CFTC regulation 4.7) will be required to receive and sign off on a disclosure document in compliance with certain CFT rules The disclosure documents contains a complete description of the principal risk factors and each fee to be charged to your account by the CTA, as well as the composite performance of accounts under the CTA's management over at least the most recent five years. Investor interested in investing in any of the programs on this website are urged to carefully read these disclosure documents, including, but not limited to the performance information, before investing in any such programs.

Those investors who are qualified eligible persons as that term is defined by CFTC regulation 4.7 and interested in investing in a program exempt from having to provide a disclosure document and considered by the regulations to be sophisticated enough to understand the risks and be able to interpret the accuracy and completeness of any performance information on their own.

RCM receives a portion of the commodity brokerage commissions you pay in connection with your futures trading and/or a portion of the interest income (if any) earned on an account's assets. The listed manager may also pay RCM a portion of the fees they receive from accounts introduced to them by RCM.

See the full terms of use and risk disclaimer here.

logo