Defining consensus: a systematic review recommends methodologic criteria for reporting of Delphi studies
Related Experiment Video
Updated: May 2, 2026

Qualitative and Quantitative Validation of Tools with Rating Scales Aimed at Assessing the Quality of University Service-Learning
Published on: August 29, 2025
Ivan R Diamond1, Robert C Grant2, Brian M Feldman3
1Child Health Evaluative Sciences Program, The Hospital for Sick Children, 555 University Avenue, Toronto, ON M5G 1X8, Canada; Department of Medical Imaging, The University of Toronto, 263 McCaul Street, 4th Floor, Toronto, ON M5T 1W7, Canada; Institute of Health Policy Management and Evaluation, The University of Toronto, 155 College Street, 4th floor, Toronto, ON M5T 3M7, Canada.
Objective:
To investigate how consensus is operationalized in Delphi studies and to explore the role of consensus in determining the results of these studies.
Study Design And Settings:
Systematic review of a random sample of 100 English language Delphi studies, from two large multidisciplinary databases [ISI Web of Science (Thompson Reuters, New York, NY) and Scopus (Elsevier, Amsterdam, NL)], published between 2000 and 2009.
Results:
About 98 of the Delphi studies purported to assess consensus, although a definition for consensus was only provided in 72 of the studies (64 a priori). The most common definition for consensus was percent agreement (25 studies), with 75% being the median threshold to define consensus. Although the authors concluded in 86 of the studies that consensus was achieved, consensus was only specified a priori (with a threshold value) in 42 of these studies. Achievement of consensus was related to the decision to stop the Delphi study in only 23 studies, with 70 studies terminating after a specified number of rounds.
Conclusion:
Although consensus generally is felt to be of primary importance to the Delphi process, definitions of consensus vary widely and are poorly reported. Improved criteria for reporting of methods of Delphi studies are required.
More Related Videos
06:16Involving Individuals with Developmental Language Disorder and Their Parents/Carers in Research Priority Setting
Published on: June 6, 2020
08:36Author Spotlight: Evaluating the Adjuvant Efficacy and Safety of Angong Niuhuang Pill in Viral Encephalitis Treatment
Published on: April 19, 2024
Related Concept Videos
Data Validation
Key parameters for method validation include:
Methods of Documentation V: CBE
In CBE, healthcare professionals establish predefined standards of practice that define what constitutes...
Decision Making: Traditional Method
First, a specific claim about the population parameter is decided based on the research question and is stated in a simple form. Further, an opposing statement to this claim is also stated. These statements can act as null and alternative hypotheses, out of which a null hypothesis would be a...
Types of Biopharmaceutical Studies: Controlled and Non-Controlled Approaches
Non-controlled studies, commonly employed for initial exploration, lack a control group, rendering them susceptible to biases and external influences. In contrast,...
Ethics in Research
Groupthink