The Un-Wisdom Of Crowds: Prediction Markets Failed Their Midterm Exams

James Surowiecki’s 2005 best-seller, The Wisdom of Crowds, produced a Malcolm-Gladwell-like bubble in enthusiasm for the strange power of the hive-mind to outguess any mere individual, expert or otherwise. Surowiecki’s premise is simple:

• “Groups are remarkably intelligent, and are often smarter than the smartest people in them.”

He developed this claim out of a charming anecdote. In 1906, the famous British statistician Francis Galton had attended an agricultural fair where he observed a contest being held to guess the weight of an ox – and as the story goes:

• “A fat ox having been selected, competitors bought stamped and numbered cards, for 6d. each, on which to inscribe their respective names, addresses, and estimates of what the ox would weigh after it had been slaughtered and ‘dressed.’ Those who guessed most successfully received prizes.”

In Surowiecki’s retelling, the message is elevated to a Great Truth.

• “Eight hundred people tried their luck. They were a diverse lot. Many of them were butchers and farmers, but there were also quite a few who had no insider knowledge of cattle. ‘Many non-experts competed,’ Galton wrote later…The crowd guessed 1,197 pounds; after it had been slaughtered and dressed the ox weighed 1,198 pounds. In other words, the crowd’s judgment was essentially perfect…”

Galton himself saw a political moral in the story.

• “The average competitor was probably as well fitted for making a just estimate of the dressed weight of the ox, as an average voter is of judging the merits of most political issues on which he votes.”

Others, especially those who apply this to the financial markets, turned Surowiecki’s and Galton’s thesis into a general argument for the superiority of collective over individual judgment.

• “Large crowds are collectively smarter than individuals. Collective knowledge and opinions of a group are better at decision-making, problem-solving, and innovating than an individual.” – Investopedia

There is even a kind of engineering-flavored explanation, drawn from signal processing techniques in which (assumed random) deviations, or bias, in individual measurements, fluctuating around a central (correct) value, can be canceled out.

• “The viewpoint of an individual can inherently be biased, whereas taking the average knowledge of a crowd can result in eliminating the bias or noise to produce a clearer and more coherent result.”

In fact, this is the same general principle underlying the standard arguments in favor of portfolio diversification, claims for the accuracy of prices in “efficient” markets, and many other aspects of orthodox finance theory: fluctuations and biases can be controlled (and ignored) by assuming they are random in nature (uncorrelated).

Financial markets are seen by some as an embodiment of the Wisdom of Crowds (WOC) principle. Millions of investors, all jostling their diverse expectations, understandings and forecasts together, are said to be able to arrive collectively at the most-nearly correct value for any widely traded asset (e.g., shares of public company equity), integrating all past, present, and future information into a preternaturally accurate share price. Thus, WOC is the philosophical mechanism underpinning Efficient Market Theory. In addition, the stock market is said to be able, through the same mechanism, to predict future macro-economic events such as recessions or recoveries (as described in an earlier column).

The media generally endorses the WOC premise, and reviewers have gushed praise for Surowiecki’s book:

• “He musters ample proof that the payoff from heeding collective intelligence is greater than many of us imagine.” – BusinessWeek
• “The impressive power of collective thinking provided here is fascinating, and oddly comforting.” – Detroit Free Press

Michael Lewis (Moneyball etc.) loved the book, too. Even Malcolm Gladwell weighed in

• “The most brilliant book on business that I’ve read in years.”

From the start of the WOC bubble, however, there have been skeptics. The Financial Times was guarded in its original review:

• [The book] is packed with amusing ideas that leave the reader feeling better-educated.”

Before long contrary assessments began to appear. Collective judgment didn’t always seem to work its magic. The WOC thesis became a foil for revisionists: e.g., “When the Crowd Isn’t Wise” (NY Times headline) and “When Crowds Aren’t Wise” (Harvard Business Review headline).

Nevertheless, pro or con, WOC is now established as a journalistic meme. It has also helped to promote an interesting new cottage industry: prediction markets.

Prediction Markets

The idea behind a prediction market is to automate Galton’s ox-guessing contest, and apply it to estimating or predicting various things, especially the outcomes of political contests. Trading on such a platform is akin to betting. Some of the earliest examples were hosted on sports-betting sites in the UK. The Defense Department famously proposed to create a forum to allow traders to bet on political events in he Middle East, including coups, wars, and terrorist incidents, as a way to marshal WOC in service of the War on Terror. (This resulted in rather ugly optics – “The idea of a federal betting parlor on atrocities and terrorism is ridiculous and grotesque” [said one U.S. Senator] – and the idea was shot down.)

Apparent early “successes” prompted academics to study the phenomenon. Prediction markets began to focus on political contests – e.g., for predicting the outcome U.S. presidential elections. The Iowa Electronic Market (IEM) became famous for its election predictions, said to outperform traditional polling. By positioning the alternatives — “Democratic Win” vs “Republican Win” – as though they were stocks to be bought and sold, the Iowa exchange produced surprisingly accurate predictions. Starting in 1988, the IEM beat the pollsters decisively over the next several Presidential elections.

PredictIt is a project developed by Victoria University in Wellington, New Zealand, which calls itself “a unique and exciting real money site that tests your knowledge of political events by letting you trade shares on everything from the outcome of an election to a Supreme Court decision to major world events.” They claim to serve 80,000 traders, and provide anonymized data used by more than 200 academic researchers and university educators. PredictIt has operated their election markets for the past 8 years under a No Action letter granted by the Commodity Futures Trading Commission (CFTC).

PredictIt covered the American midterm elections intensely, allowing traders to bet on almost every race and scenario. Their user interface far outdoes Iowa’s in gloss and granularity. (Who knew the Kiwi’s were so interested in the Georgia Senate contest, e.g.?) However, the CFTC has recently moved to restrict and perhaps prohibit PredictIt from operating in the U.S. after February. PredictIt has sued the CFTC to block this action.

Kalshi is a start-up company, in beta today, which operates a series of markets keyed to various events. such as certain legislation actions, supreme court decisions, tax changes, and the like. The company’s banner portrays the range of its projected offerings.

Kalshi has also applied to the CFTC to be permitted to trade election contracts, which would look something like options or futures based on traditional commodities (hence the CFTC’s assumption of regulatory jurisdiction), described as “cash-settled, binary contracts based on the question such as: “Will be in control of the ?”

The CFTC has seemed interested but hesitant. The Commission issued a formal Request for Public Comment on this matter in August, with a goal to issue a ruling in late October. Yet according to Bloomberg, based on objections of its staff, the Commission is apparently “poised to deny” approval of the election contracts – at least for now.

Polymarket – apparently a start-up based in New York – describes itself as “an information markets platform that harnesses the power of free markets to demystify real world events.…” Polymarket portrays itself as more technically au courant than its competitors: its exchange is based on blockchain technology, and it allows users to transact in crypto-tokens – USD Stablecoins – rather than “real money.”

• “On Polymarket, you build a portfolio based on your forecasts and earn a return if you are right. When you decide to buy shares in a market, you are weighing in with your own knowledge, research, and view of the future. Market prices reflect what traders think are the odds of future events, turning trading activity into actionable insights that help people make better decisions. As a result, Polymarket is a leading source of unbiased and real-time data about future events.”

Gnosis is another Fintech-flavored start-up which claims to be building a “permissionless prediction market” (whatever that is). There is not much information available on their efforts so far.

Something of a mini-bubble in technologize WOC is developing, despite regulatory headwinds. Prediction markets have acquired a reputation for uncanny accuracy which has boosted the moral standing of the WOC thesis. As a WSJ commentator wrote, indignantly, just last week, arguing against the CFTC’s move to clamp down on PredictIt –

• “It’s a blow to the public at large, because political futures have proven to have better predictive power than polls….That would be unfortunate for liberty. If investors can express their opinions on the future prices of corn and pork bellies, surely the First Amendment also protects their ability to do the same on elections and other political matters. ”

So – do they really work? Do these platforms reliably produce better forecasts than other methods?

Prediction Markets Failed Their Midterm Test, Big-Time

Unfortunately, the 2022 midterm elections did not turn out well for any of the operational prediction markets. They got the key calls all wrong.

Control of the Senate

Right up to election day, all the prediction markets forecast a clear…