Interventions to Improve the Quality of Multiple-Choice Questions: A Systematic Review and Meta-Analysis

Area of Science:

• Medical Education Research
• Assessment Quality Improvement
• Health Professions Education

Background:

• High-quality multiple-choice questions (MCQs) are essential for valid and reliable assessments in health professions education.
• Interventions to improve MCQ quality are crucial for effective evaluation of learner competency.

Purpose of the Study:

• To identify, review, and synthesize literature on interventions for improving MCQ quality.
• To evaluate the effectiveness of faculty training and peer review in enhancing MCQ item quality.

Main Methods:

• Systematic literature review and synthesis of interventions.
• Meta-analysis of quasi-experimental studies assessing item quality using objective metrics (e.g., discrimination index, difficulty, distractor efficiency).

Main Results:

• Both faculty training and peer review improve MCQ quality, with peer review showing a greater effect (Cohen's d = 1.68 vs. 0.98).
• Longitudinal faculty training workshops are more effective than brief, one-time sessions.
• Resource-efficient approaches like self-paced modules and targeted reviews are promising.

Conclusions:

• Peer review and ongoing faculty development are vital for sustainable MCQ quality assurance.
• Institutions should adopt evidence-based, resource-conscious strategies for faculty development and peer review.
• Improving MCQ quality is critical for valid and reliable health professions education assessments.