Delphi Method

The Delphi method gathers expert opinions through questionnaires answered privately. Experts see a summary of the group's answers and can reconsider their views in later rounds. This can help people learn from one another, but it guarantees neither agreement nor accuracy. Try this: Before using it, write down when the rounds will stop.

Imagine a team estimating when a new technology will become widely used. This is a hypothetical example of a forecasting question where suitable historical data may be scarce. A facilitator recruits people with relevant knowledge and sends questionnaires over several rounds. Participants' answers are presented without identifying them to one another; this is the method's anonymity feature. [1]

Between rounds, the facilitator provides controlled feedback: a prepared summary of participants' judgments. It usually includes an average or another central estimate and may include reasons for particular answers. Participants review the feedback and may revise their estimates. The final judgments are combined numerically, often using an average. [1]

The theoretical purpose of private responses is to reduce pressure from dominant people, status and the majority. Repeated rounds also let participants change their minds without publicly defending an earlier position. These are proposed mechanisms, not guarantees that influence disappears or judgments become correct. As practical guidance, specify who can access identities and responses rather than promising complete privacy. [1]

Before starting, set an explicit stopping rule: for example, stop when changes between rounds fall below a stated threshold, with a maximum number of rounds. This is practical guidance, not a proven optimal rule. If disagreement persists, report the separate estimates and examine their evidence and assumptions. Stable answers mean little change; they do not necessarily mean agreement. Ask participants to distinguish supporting evidence from personal judgment.

Rowe and Wright found no consistent superiority over other structured group methods. Their review also cautioned that study designs varied, limiting generalization. Treat Delphi as a way to organize expert judgment, not a guarantee of agreement or accurate forecasts. [1]

• Empirical evidence: Rowe and Wright reported that comparisons favored Delphi over groups whose independent estimates were simply averaged in 12 studies, versus two favoring those groups and two ties. Comparisons with ordinary interacting groups favored Delphi in five studies, versus one favoring interacting groups and two ties. These counts do not establish a guaranteed advantage. [1]
• Comparative limits: The review found no consistent evidence that Delphi outperformed other structured group procedures. Differences in participant expertise, tasks and feedback, together with simplified laboratory implementations, limited generalization. [1]
• Theory and mechanism: The review describes anonymity as intended to reduce pressure from status, dominant individuals and majorities. Repeated questionnaires allow reconsideration, while controlled feedback shares judgments and sometimes reasons; these proposed benefits do not establish automatic accuracy. [1]

The review critiques the quality of the evaluation evidence, including missing design details, limited statistical analysis and inadequate control of participant expertise and feedback. Its reported comparisons should therefore be read cautiously. The excerpt provides no basis for a fixed optimal panel size or a universally best number of rounds. As practical guidance, select relevant expertise, explain how feedback is prepared, assess the evidence behind estimates and define stopping rules beforehand. Preserve persistent disagreement and its reasons rather than treating a single average as proof of agreement. Specify privacy arrangements explicitly; participant anonymity is not a promise of complete confidentiality.