Research · Essay
Forecasting Without Prediction
A point estimate is a distribution with the interesting parts removed.
By NDI Research ✦ ✦4 minutes’ reading
In Brief
- A prediction states what will happen. A forecast states how likely each thing is. Only the second can be scored honestly, and only the second can be acted on.
- Two forecasters can agree on the expected outcome and disagree completely about risk. The disagreement lives in the tails, which point estimates delete.
- We score every forecast with a proper scoring rule. It rewards being right about uncertainty, not about outcomes.
Ask most systems about the future and they will give you a number. The index will be 4% higher. Inflation will print at 2.9. The spread will tighten by twenty basis points. The number is precise, easy to communicate, and almost always wrong. More importantly, it is wrong in a way that cannot be measured: when the index ends 1% higher, nobody knows whether the forecaster was unlucky or incompetent.
We do not ask our models for numbers of this kind. We ask them for probabilities over explicit outcomes, and we hold them to account for those probabilities. This is the difference between prediction and forecasting, and it is the foundation on which everything else we build sits.
§ IWhat a point estimate hides
Consider a single question: how will a broad equity index move over the next month? The consensus estimate, derived from analyst expectations, is a gain of roughly 1%. Our own models, aggregated, also produce an expected gain of roughly 1%. On the surface, there is no disagreement.
- Consensus
- NDI
Underneath, the two views are very different. Consensus places 55% of its weight on a quiet month. We place 40% there, and assign almost three times as much probability to each tail. If you act on the point estimate, you are acting on the one feature the two distributions share. Everything that matters for sizing, hedging and survival has been thrown away.
This is not an edge case. It is the normal condition. The mean is the least informative moment of most distributions that matter in markets, and the moment on which disagreement is least likely to exist, because it is the moment everyone is asked about.1
§ IIFrom numbers to questions
Distributions over continuous outcomes are useful but hard to reason about. In practice, we decompose a period into a set of binary questions with fixed resolution criteria: Does liquidity expand through October? Does realized volatility exceed implied? Each question has a date, a source and an unambiguous answer.
Binary questions have three advantages. They force the model to commit to a statement about the world rather than a vague tendency. They can be compared directly with market-implied or survey-derived probabilities, which gives us a reference for every forecast we make. And they are scorable: at resolution, each forecast either earned its probability or it did not.
§ IIIScoring honesty
A scoring rule is proper if the forecaster minimises expected penalty by reporting what they actually believe. The rule we use most is the Brier score, the mean squared difference between the probability assigned and the outcome:
Under the Brier score, saying 90% when you believe 70% is penalised in expectation, and so is saying 50% to avoid embarrassment. There is no rhetorical position that beats honesty. That property matters more than it appears. Most institutional forecasting is scored by narrative — the analyst who called the turn is remembered, the forty who did not are forgotten. A proper score has no memory for stories.
| Forecast | Outcome | Score | Reading |
|---|---|---|---|
| 90% | YES | 0.010 | Confident and right |
| 63% | YES | 0.137 | Leaning and right |
| 50% | either | 0.250 | No information |
| 63% | NO | 0.397 | Leaning and wrong |
| 90% | NO | 0.810 | Confident and wrong |
§ IVPricing futures
The output of this process is not a view on what will happen. It is a set of prices on what might. A probability of 63% on a liquidity expansion is, functionally, a statement that we would be indifferent to paying 63 cents for a contract that pays a dollar if liquidity expands. When the market offers that contract at 42 cents, we have something to talk about.
That is the entire idea in one sentence. We do not claim to know the future. We price possible futures, compare our prices with everyone else's, and act only where the difference is large, justified and executable. The rest of this research series is about each of those three words.