Extremizing Forecasts

Extremizing means moving a combined probability forecast away from 50% toward 0% or 100%. It can help when combining different evidence produces a forecast that is too cautious, but agreement alone does not show that it will help. Check this: Before adjusting a forecast, check whether the experts used overlapping evidence.

To extremize forecasts, first combine individual probabilities, for example by averaging them, then move the result farther from 50%. An average above 50% moves upward; an average below 50% moves downward. The direction of this adjustment does not establish whether it improves accuracy.

As a hypothetical example, five product managers each give a feature a 65% chance of success. Their average is also 65%. If they contribute useful different customer evidence, their combined information could justify greater confidence. If they all rely on the same survey, their agreement adds less information. Neither agreement nor working separately establishes the right adjustment.

The supplied paper proposes two mechanisms. First, probabilities have boundaries: near 100%, an error has little room to move upward but much more room to move downward. Under the paper's error assumptions, this can pull an average toward 50%. Second, individuals may appropriately temper their confidence because they know they lack information. Combining different evidence can then justify more confidence than a typical individual expresses. This is a theoretical explanation, not a claim that all forecasters are excessively cautious. [1]

The paper reports that extremizing improved forecasts of international events, measured using the Brier score, a measure of probability prediction error where lower scores are better. It also notes cases where making forecasts less extreme helped, possibly because the original judgments were too extreme. Information overlap, biased evidence and existing overconfidence therefore matter; independence and agreement do not guarantee improvement. Even all available information can leave uncertainty. [1]

As practical guidance, use backtesting: compare candidate adjustments against outcomes of past forecasts. Choose the adjustment using one set of resolved questions, then evaluate it on held-out questions, meaning questions excluded from that choice. Check calibration—whether events assigned similar probabilities occur about that often—and overall prediction error against the unchanged average. Use an adjustment only when that evaluation supports it for comparable forecasts; otherwise, leave the average unchanged.

• Empirical evidence: Baron and colleagues report that extremizing improved aggregated forecasts of international events as measured by Brier scores. They also mention subsequent cases where making judgments less extreme was preferable, possibly because the initial judgments were too extreme. The excerpt does not establish a universal benefit [1]
• Theory: The paper proposes that the boundaries of the probability scale can pull an arithmetic average toward 50% under its error assumptions. Near a boundary, upward and downward errors have different amounts of room; the paper distinguishes this effect from the separate missing-information mechanism [1]
• Theory: Individuals may reasonably reduce confidence to account for information they lack, while a group combines different evidence. The authors explicitly explain that this individual caution need not be a bias, and that overconfidence can reduce or reverse the proposed effect [1]
• Model assumptions and limits: The argument assumes that information errors are equally likely to favor either direction relative to the estimate based on all available information. The authors describe their model as approximate, identify information sharing as relevant to the group's information advantage, and allow uncertainty to remain even with all available information [1]

The excerpt is truncated and does not establish a universal adjustment strength or a rule linking greater agreement to greater extremizing. Working separately does not ensure that forecasters use different evidence, and shared evidence does not establish that every adjustment will fail. The proposed mechanisms depend on assumptions about information and errors; they are not guarantees for a particular forecast. Held-out backtesting and calibration checks are practical guidance here, not a reported finding from this excerpt. Their usefulness also depends on having enough resolved questions that reasonably resemble the forecasts being adjusted.