Between 1984 and 2003, Philip Tetlock, a psychologist at the University of Pennsylvania, asked 284 people who made their living commenting on or advising about political and economic trends to make predictions. Would the Soviet Union survive? Would there be a war in the Persian Gulf? Would the economy grow or shrink? He collected 28,000 predictions, recorded the experts’ confidence levels, and waited to see what happened.
The results were not flattering. On average, the experts’ predictions were barely more accurate than random chance. A dart-throwing chimpanzee, Tetlock noted, would have done about as well. More disturbingly, the experts who were most confident in their predictions tended to be the least accurate. The pundits who appeared most frequently on television and who spoke in the most definitive terms were reliably worse at forecasting than their less prominent colleagues.
Tetlock borrowed a framework from Isaiah Berlin’s essay about the fox and the hedgehog. Hedgehogs are thinkers who see the world through a single big idea. Foxes are thinkers who draw from many sources and are comfortable with ambiguity. Tetlock found that foxes, on average, made better predictions than hedgehogs. Not because foxes were smarter, but because they were more willing to update their beliefs when new evidence appeared and less attached to any single explanatory framework.
The book is academic in tone. Tetlock is a social scientist, and the methodology sections are detailed. He addresses potential objections systematically: maybe the experts were asked unfair questions, maybe the time horizons were wrong, maybe the subject matter was inherently unpredictable. He works through each objection and shows that while these factors matter, they do not explain away the core finding. Experts, as a group, are not good at predicting the future.
The implications are significant for anyone who relies on expert forecasts. Business leaders who hire consultants to predict market trends. Investors who follow analyst recommendations. Policy makers who consult think tanks. In each case, Tetlock’s research suggests that the confidence with which a prediction is delivered is a poor guide to its accuracy.
Tetlock followed this book with Superforecasting, co-written with Dan Gardner, which examined the small number of people who actually are good at prediction and what distinguishes them. But Expert Political Judgment came first and remains the foundational work. It asks an uncomfortable question and answers it with two decades of data.
