Causes of bias in Delphi studies and consensus statements: a methodological review
Related Experiment Videos
Prashant Nasa1,2,3, Ravi Jain4, Deven Juneja5
1Department of Critical Care Medicine and Anaesthesia, Royal Wolverhampton NHS Trust, New Cross Hospital, Wolverhampton, UK.
Introduction:
The Delphi methodology is a widely-used tool for achieving expert consensus in healthcare, particularly in areas where empirical evidence is limited or uncertain. Despite its structured design, the Delphi process remains vulnerable to implicit cognitive biases that may influence judgement and group decision-making. Such biases may inadvertently affect the quality of the consensus process, allow louder voices to influence outcomes, restrict inclusivity and undermine the validity, transparency and evidence-based nature of recommendations.
Methods:
We conducted a methodological review synthesising literature from cognitive psychology, behavioural science and methodological research on consensus development and Delphi processes. Using a concept-mapping approach, established cognitive biases were mapped to different stages of the Delphi process, including study design; panel selection; survey development; iterative scoring; and consensus interpretation.
Results:
Ten cognitive biases were organised into four domains reflecting different levels of influence within consensus development: steering-group and design-level biases, panel composition and contextual biases, social influence and conformity biases; and individual cognitive heuristic biases. A conceptual framework was developed to systematically identify these biases within Delphi-based consensus studies. Examples illustrating how these biases may arise in healthcare Delphi processes are presented, alongside potential mitigation strategies.
Discussion:
Recognition and mitigation of cognitive biases are essential for strengthening the methodological rigour and credibility of Delphi studies. Strategies to reduce bias include diversifying expert panels; using neutral language in survey statements; maintaining anonymity during voting; involving methodological expertise in Delphi design; and ensuring transparent iterative feedback. The framework needs prospective validation before its widespread use and may assist journal editors and peer reviewers in critically appraising Delphi studies and improving reporting standards in consensus research.
Related Concept Videos
Bias
In statistics, a sampling bias is created when a sample is collected from a population, and some members of the population are not as likely to be chosen as others (remember, each member...
Bias in Epidemiological Studies
Types of Biopharmaceutical Studies: Controlled and Non-Controlled Approaches
Non-controlled studies, commonly employed for initial exploration, lack a control group, rendering them susceptible to biases and external influences. In contrast, controlled...
Confirmation Biases
Correspondence Bias
Blind Procedures