55 pages • 1-hour read
Superforecasting: The Art and Science of Prediction
A modern alternative to SparkNotes and CliffsNotes, SuperSummary offers high-quality Study Guides with detailed chapter summaries and analysis of major themes, characters, and more.
Chapters 8-9Chapter Summaries & Analyses
Chapter 8 Summary: “Perpetual Beta”
Superforecasters tend to possess what the psychologist Carol Dweck calls “a growth mindset” as opposed to a fixed mindset; they believe that they can improve their potential as long as they work hard. Those with a growth mindset attend to new information that can increase their skills and knowledge.
A growth mindset is invaluable to a superforecaster. To obtain skill in forecasting, one must combine trial and error to scrutinize their technique. Without such scrutiny, experience can only take a person so far. One reason why forecasters struggle to obtain sufficient feedback is ambiguous language, whereby vague terms like “probably” and “likely” are deployed instead of numbers (181). If a forecast turns out to be wrong, such unclear language makes it difficult to go back and learn from errors. Another obstacle is hindsight bias, whereby learning the outcome of a situation distorts our perspective of what we knew before; this bias is a person’s belief that they “knew it all along” when they did not, in fact, know it all along. This bias makes it difficult to learn from mistakes. Still, like Dweck’s growth-minded subjects, most superforecasters are eager for feedback. Their willingness to continually improve makes them akin to computer programs that have perpetual beta, or trial mode. Perpetual beta “is not intended to be released in a final version but will instead be used, analyzed, and improved without end” (190).
Of all the traits that make a superforecaster, perpetual beta is arguably the most important. Indeed, the commitment to self-improvement is three times as crucial as the next front-runner, intelligence.
Chapter 9 Summary: “Superteams”
One key question at the Good Judgment Project headquarters was whether teams of forecasters outperform individuals. Tetlock was aware of his doctoral adviser Irving Janis’s 1972 definition of “groupthink,” the phenomenon whereby group members develop mutual illusions as a form of bonding. Groupthink destroys critical thinking, and as a result, the group’s accuracy and efficacy diminish. Groupthink can occur regardless of members’ intelligence or expertise, especially when the group has largely uniform views.
However, the advantage of groups is that information and perspectives can be shared. As shown in earlier chapters, aggregation is key to accuracy. However, a spirit of questioning, both of oneself and one’s team members, is also essential. The GJP researchers were curious as to whether groupwork would boost superforecasters’ power or diminish it. It was important to test this because, in the outside world, many decisions are made by groups rather than individuals. Testing groups took different formats as the GJP developed. In the first year, 2011-2012, before any superforecasters had been identified, the GJP randomly split some forecasters into groups, while others worked individually. Those working in groups were instructed to be critical of each other’s forecasts and offer precise constructive feedback. At the end of this pilot year, the results showed that teams were 23% more precise than individuals. In the second year, 2012-2013, the GJP created teams of superforecasters to see what would happen when these newly anointed individuals were put together. The GJP was concerned that complacency or groupthink might impair superforecasters’ formerly superior judgment. However, the superforecaster teams overall welcomed pushback and established a psychologically safe atmosphere where it was helpful to give constructive feedback.
With regard to economic questions, the superforecasters even outperformed prediction markets, or “markets that trade in predictions, meaning traders buy and sell contracts on specified outcomes” (206). One reason for the superforecasters’ success was that a large proportion of the teams was generous with their time, expertise, and skills. Additionally, diversity of perspective was a crucial component that enabled the teams to think critically. Tetlock notes that the superteams were compiled based on ability rather than diversity and that organizations should exercise caution in dividing the highest performers into teams, as it could create division and conflict.
Chapters 8-9 Analysis
Chapter 7 emphasized that an openness to new information is requisite of forecasters as they seek to keep up with the news, and this openness is a continued motif in Chapters 8 and 9, which discuss superforecasters’ willingness to challenge their individual judgment through peer feedback and groupwork. Just as superforecasters are typically averse to the System-1-style thinking of dogmatism or confirmation bias, GJP’s top performers eschew a fixed mindset, the belief that intelligence is predetermined with rigid limitations. Superforecasters are eager to update their skills—and update themselves. The term “perpetual beta,” which stems from information technology, indicates superforecasters’ machine-like detachment from particular processes or ideologies and reflects their foxlike determination to learn new skills.
Though the book is scientific and highly expository, the authors avoid an overly aloof tone by using humanizing narratives to illustrate principles. For example, Tetlock and Gardner bring life to the idea of a growth mindset by sharing superforecasters’ personal stories of overcoming adversity, including layoffs and health challenges. Such personalizing elements make the text’s abstract concepts more vivid and comprehensible, but these stories also drive home the point that grit, perseverance, and self-awareness are what prevent superforecasters from becoming complacent or intellectually lazy. Still, as with “updating” their beliefs (described in Chapter 7), superforecasters can take feedback too far when they overcorrect their forecasting methods. After receiving corrective feedback, they might view their errors as bigger problems than they are, and this could distort how they approach predictions in the future. This tightrope walk—the precarious balance between under- and overcorrection—demonstrates that just as superforecasting is not a simple formula, superforecasters are not simple people, nor are they superhuman.
The theme of Hedgehogs and Foxes emerges with a twist in the authors’ analysis of groupwork. Arguably, a team of superforecasters is the ultimate fox, as they are a composite of the minutiae known by multiple curious minds. However, in practice, the team’s predictions are superior only when the forecasters do not unconsciously mimic each other’s predictions for the sake of the group unity or freeload off the efforts of their companions. Even in group settings, they must remain individuals, confident enough in their original judgments to argue for them and enter hearty negotiations. Here, it is the composite of forecasters’ traits that makes them so skilled. The idea of a composite, both in individuals’ traits and in group work, indicates that there are elements of multiplicity and randomness that enhance a forecasting team. This eclecticism reflects the complex nature of the events that shape our world, as multifaceted individuals are the best qualified to investigate its present and future.
Unlock all 55 pages of this Study Guide
Get in-depth, chapter-by-chapter summaries and analysis from our literary experts.
- Grasp challenging concepts with clear, comprehensive explanations
- Revisit key plot points and ideas without rereading the book
- Share impressive insights in classes and book clubs