Related Experiment Video
Updated: Feb 27, 2026
Qualitative and Quantitative Validation of Tools with Rating Scales Aimed at Assessing the Quality of University Service-Learning
Published on: August 29, 2025
The Use of the Delphi and Other Consensus Group Methods in Medical Education Research: A Review
Susan Humphrey-Murto1, Lara Varpio, Timothy J Wood
1S. Humphrey-Murto is associate professor, Department of Medicine, Faculty of Medicine, University of Ottawa, Ottawa, Ontario, Canada. L. Varpio is associate professor, Department of Medicine, Uniformed Services University of the Health Sciences, Bethesda, Maryland. T.J. Wood is associate professor, Department of Innovation in Medical Education, Faculty of Medicine, University of Ottawa, Ottawa, Ontario, Canada. C. Gonsalves is assistant professor, clinician educator, and clinical hematologist, Department of Medicine, Faculty of Medicine, University of Ottawa, Ottawa, Ontario, Canada. L.-A. Ufholz is a medical librarian, Faculty of Medicine, University of Ottawa, Ottawa, Ontario, Canada. K. Mascioli is psychiatrist, Department of Psychiatry, and education fellow, Department of Innovation in Medical Education, Faculty of Medicine, University of Ottawa, Ottawa, Ontario, Canada. C. Wang is an internal medicine resident, Department of Medicine, Faculty of Medicine, University of Ottawa, Ottawa, Ontario, Canada. T. Foth is assistant professor, School of Nursing, Faculty of Health Sciences, University of Ottawa, Ottawa, Ontario, Canada.
Purpose:
Consensus group methods, such as the Delphi method and nominal group technique (NGT), are used to synthesize expert opinions when evidence is lacking. Despite their extensive use, these methods are inconsistently applied. Their use in medical education research has not been well studied. The authors set out to describe the use of consensus methods in medical education research and to assess the reporting quality of these methods and results.
Method:
Using scoping review methods, the authors searched the Medline, Embase, PsycInfo, PubMed, Scopus, and ERIC databases for 2009-2016. Full-text articles that focused on medical education and the keywords Delphi, RAND, NGT, or other consensus group methods were included. A standardized extraction form was used to collect article demographic data and features reflecting methodological rigor.
Results:
Of the articles reviewed, 257 met the inclusion criteria. The Modified Delphi (105/257; 40.8%), Delphi (91/257; 35.4%), and NGT (23/257; 8.9%) methods were most often used. The most common study purpose was curriculum development or reform (68/257; 26.5%), assessment tool development (55/257; 21.4%), and defining competencies (43/257; 16.7%). The reporting quality varied, with 70.0% (180/257) of articles reporting a literature review, 27.2% (70/257) reporting what background information was provided to participants, 66.1% (170/257) describing the number of participants, 40.1% (103/257) reporting if private decisions were collected, 37.7% (97/257) reporting if formal feedback of group ratings was shared, and 43.2% (111/257) defining consensus a priori.
Conclusions:
Consensus methods are poorly standardized and inconsistently used in medical education research. Improved criteria for reporting are needed.
More Related Videos
14:43A Novel Method for Involving Women of Color at High Risk for Preterm Birth in Research Priority Setting
Published on: January 12, 2018
13:44Project-Based Learning Guidelines for Health Sciences Students: An Analysis with Data Mining and Qualitative Techniques
Published on: December 9, 2022
Related Concept Videos
01:56Ethics in Research
01:34Groupthink
01:30Ethical Dilemmas II
01:18The Scientific Method in Nursing Process
When using research findings to change practice, one must understand the process used to guide a study. The scientific method is a systematic, step-by-step process that supports the data's validity, reliability, and generalizability. As a result, findings can be...
01:26Methods Of Healthcare Delivery System
Managed Care System:
The managed care system is designed to control the cost while maintaining the quality of care. The patient's care from admission to discharge is planned by the primary care provider or the case manager, also known as the gatekeeper. In a managed care system, the number of care providers is...
01:23Types of Biopharmaceutical Studies: Controlled and Non-Controlled Approaches
Non-controlled studies, commonly employed for initial exploration, lack a control group, rendering them susceptible to biases and external influences. In contrast,...