- Library
- Data, Logic & Reasoning
- Forecasting & Calibration
- Aggregation & Wisdom of Crowds
- Conditions for Crowd Wisdom
Mental model
Conditions for Crowd Wisdom
An editorial checklist for collecting and combining judgments: varied information, private first answers, local knowledge and suitable aggregation. It does not guarantee accuracy.
Discover
A manager wants each engineer's first completion-time estimate before they see anyone else's answer. A few engineers have replied, so a partial team average is available. Should the manager show it to engineers who have not yet submitted?
When should the team see feedback?
Learn how to collect initial judgments and check whether they add useful information.
Understand
Understand
Groups can make better decisions when people contribute useful information from different sources. Private first answers can help preserve those differences, but people may still share the same mistaken evidence. Try this: Collect private estimates and ask what evidence each person used before discussing the results.
Full explanation
Full explanation
For this lesson, use an editorial checklist with four checks: diversity, meaning different relevant information or perspectives; independence, meaning judgments are not simply copied from others; decentralization, meaning people can contribute local knowledge; and aggregation, meaning a rule for combining judgments. Treat these as practical guidance, not as conditions that are always required or that guarantee a correct answer.
Combining judgments can help when individual mistakes offset one another. But correlation—errors tending to move together—can limit that benefit. People answering privately may still rely on the same faulty report. The paper discusses a classical majority-vote result that assumes statistically independent guesses, each with a probability greater than one-half of being correct. This is a specific theoretical assumption, not something established merely by collecting answers separately. [1]
Kao and Couzin examine theoretical models in which individuals use environmental cues, meaning signals that provide information, and a majority vote selects between two options. In some modeled environments, small groups outperform larger ones because random variation in which cues members use and what they observe can sometimes prevent shared information from controlling the decision. Results depend on cue reliability, correlation and voting strategy; the models establish no universal best group size. These are model results, not human field evidence for the four checks. [1]
For a hypothetical project forecast, ask engineers privately for completion estimates and their reasons. An online poll that hides responses until submission records initial answers before participants see others' responses. If estimates concern the same scope and use the same units, an average can provide a starting summary. Check shared assumptions and relevant expertise before relying on it: averaging alone does not establish accuracy.
Private first answers need not mean permanent secrecy. The practical collection procedure suggested here is to record an initial answer before showing anyone else's views, then make any later discussion a separate step. This is an editorial suggestion for keeping track of how judgments change, not a result showing that a particular forecasting method is universally best.
Research
Research
-
Theory: Kao and Couzin model collective choices using environmental cues and majority voting. Whether accuracy keeps improving with group size depends on cue reliability, information correlation and the strategy individuals use to select cues. [1]
-
Model mechanism: In parts of the multiple-cue model, random variation in small groups can reduce the control of correlated information and improve accuracy relative to larger groups. This advantage is conditional, not a general recommendation to add randomness. [1]
-
Scope and evidence: The second model, involving fluctuations in cue reliability, often favors either a solitary individual or a very large group within the parameter space examined. The excerpt also discusses earlier animal laboratory experiments; its theoretical findings do not constitute human field validation of the four-part checklist. [1]
Limitations
Limitations
The supplied evidence concerns simplified models and discussion of earlier studies. It does not establish four necessary-and-sufficient conditions, a universal group-size cutoff, or a ranking of hidden polls, Delphi rounds and markets. Private responses prevent exposure to others' answers during collection, but do not ensure statistically independent information. Averaging is a practical option for comparable numerical estimates; its usefulness depends on information quality, shared errors and the task. The project forecast is a hypothetical teaching example, not a reported experiment.
Try it
Synthesize
Choose a pattern from the guide, then pick an action to try with it.
Which pattern stands out?
What will you try?
Choose a pattern above to select an action.
Sources
Sources
- [1] Decision accuracy in complex environments is often maximized by small group sizesAlbert B. Kao and Iain D. Couzin - 2014
Try it
Check your understanding
A team wants to record initial judgments before participants see any other participants' responses or group feedback. Which setup explicitly meets that goal?
Show the guide's explanation
Answer: An online poll where responses are hidden until you vote
The hidden poll records each initial answer before showing other responses. Each other setup supplies another person's estimate or earlier group feedback first. This meets the stated collection goal without proving greater accuracy or eliminating shared errors.
Keep exploring
Find another idea for the decision in front of you.
The complete Reframo library is free to read. Explore another guide whenever you are ready.