Forecasting and Its Limits

Forecasting is everywhere. Weather apps give you a 60 percent chance of rain. Financial analysts predict stock prices. Sports commentators estimate which team will win the championship. Scientists model when sea levels will reach certain heights. And AI researchers try to predict when machines will reach various capabilities. Forecasting is one of the most important — and most humbling — intellectual activities humans engage in.

What Forecasting Actually Is

A forecast is a specific, stated prediction about something that will or will not happen by a certain time. The best forecasts include three things: a clear description of the event, a time horizon (when), and a probability (how likely). For example: there is an 80 percent chance that AI will win a gold medal at the International Math Olympiad by 2028. Having a probability is crucial. If you say AI will definitely change education, you are making a vague claim that is almost impossible to test. If you say there is a 60 percent chance that AI tutoring tools will be used by more than half of all US public school students by 2030, that is a forecast — concrete, dated, and checkable.

Why Forecasting AI Is Especially Hard

Forecasting the future of AI is notoriously difficult, and the track record is sobering. In the 1950s, some of the brightest minds in computing predicted that machines would be able to think like humans within ten years. It has been seven decades and the question is still contested. In the early 2020s, many experts said large language models could never hold a coherent conversation — and within two years those same models were passing bar exams. Why is it so hard? Several reasons stack up. AI progress is discontinuous — it can stall for years, then leap forward in months. It depends on breakthroughs in hardware, data, and algorithms that are themselves hard to predict. It is shaped by economic incentives and political decisions that are inherently uncertain. And humans have a deep tendency to anchor forecasts on the present, underestimating how fast things can change or overestimating how fast current trends will continue.

The Superforecaster Approach

Psychologist Philip Tetlock spent decades studying who makes good forecasts. He found that most expert pundits — people who appear on television and write confident op-eds — are only slightly better than chance at predicting events in their own fields. But a small group of people, whom he called superforecasters, significantly outperformed experts. What made them different? Superforecasters constantly update their beliefs when new evidence arrives. They break big questions into smaller, more answerable ones. They think in probabilities rather than yes/no certainties. They keep score — they track their past predictions and correct for their own biases. And they are genuinely curious rather than attached to being right. These are skills anyone can develop. They do not require a Ph.D. They require discipline and honesty.

What Forecasting Cannot Do

Even the best forecasting cannot eliminate uncertainty. It cannot tell you exactly when a breakthrough will happen, or whether a new discovery will change everything. It cannot account for events that are genuinely novel — no one's model included a global pandemic in January 2019. It cannot force leaders and companies to act on its conclusions. The goal of forecasting is not certainty. The goal is calibration — having your confidence match your accuracy. A forecaster who is right 70 percent of the time on things they rate at 70 percent confidence is extremely well calibrated. That is the target: not always right, but consistently honest about how sure you are.