Less is more? A systematic review and network meta-analysis on MCQ option numbers

Multiple-choice questions (MCQs) are widely used in health sciences education due to their scalability and objectivity. While traditionally constructed with four or five options, emerging evidence suggests that using fewer options may enhance test efficiency without compromising psychometric quality. This study systematically compared the performance of 3-, 4-, and 5-option MCQs.

A systematic review and network meta-analysis (NMA) was conducted using studies from electronic databases. Included studies compared psychometric outcomes of MCQs with 3-, 4-, or 5-option formats. Outcomes included student scores, difficulty (a measure of how easy an item was, with a higher index indicating greater ease of answering) and discrimination indices, response time, reliability coefficients, and risk of non-functioning distractors (NFDs). A random-effects NMA was performed using Hedges' g and odds ratios (OR), with 95% confidence intervals. The certainty of evidence was assessed using the GRADE approach.

Forty-six studies (33,437 students and 7,535 test items) were included. Compared to 5-option MCQs, 3-option MCQs had significantly higher student scores (g = 0.42; 95% CI: 0.28, 0.56), higher difficulty index (g: 0.27; 95% CI: 0.03, 0.51), shorter test completion time (g = -1.78; 95% CI: -2.1, -1.5), and lower risk of NFDs (OR = 0.6; 95% CI: 0.4, 0.8). Discrimination indices and reliability coefficients did not differ significantly across option formats. Bootstrap and sensitivity analyses corroborate these findings. Amongst studies in health sciences, 3-option (g: 0.38; 95% CI: 0.09, 0.67) and 4-option MCQs (g: 0.29; 95% CI: 0.01, 0.57) had higher difficulty index compared to 5-option MCQs; no differences were observed in the discrimination index between the options; and 3-option MCQ had significantly lower risk of an NFD compared to 4-option (OR: 0.49; 95% CI: 0.32, 0.75) and 5-option MCQs (OR: 0.56; 95% CI: 0.37, 0.84). The evidence certainty was rated very low.

MCQs with 3-options demonstrate comparable validity and psychometric quality to those with 4- or 5-options, while offering efficiency benefits such as shorter response time and reduced risk of NFD. Given the very low certainty of evidence, these findings should be interpreted with caution. While the direction of effect appears consistent, the magnitude and generalizability remain uncertain, and educators are advised to consider local validation before widespread adoption.