Very short answer questions vs. multiple choice questions in health professions education: a systematic review and...

Multiple Choice Questions (MCQs) remain the most widely used written assessment format in health professions education due to their efficiency and reliability; however, their susceptibility to cueing raises concerns regarding their authenticity in assessing true knowledge. Very Short Answer Questions (VSAQs) have emerged as a promising alternative that reduces cueing effects and requires learners to generate responses with greater cognitive engagement, while retaining the logistical advantages of computer-marked assessments. This systematic review and meta-analysis evaluates the comparative performance, discrimination, reliability, and practice effects of VSAQs versus MCQs.

Following PRISMA guidelines (31), a systematic search identified studies comparing VSAQs and MCQs in undergraduate and postgraduate health professions education. Data extraction and risk of bias assessment were conducted independently by two reviewers, with disagreements resolved through consensus. Mean scores, standard deviations, discrimination indices, and reliability coefficients were extracted. Standardized Mean Differences (SMD) were pooled using a random-effects model. Psychometric indices were transformed using Fisher's z-scores. Heterogeneity was quantified using the I² statistic, and sensitivity and subgroup analyses were performed to explore the impact of outliers.

Six cohorts (n = 1,191) derived from three included studies contributed data for score comparisons. The initial pooled analysis showed no significant difference between VSAQs and MCQs (SMD = - 0.52; 95% CI - 1.34 to 0.30), with extreme heterogeneity (I² = 98%). Sensitivity analysis identified Dhok et al. (2023) as an outlier. Excluding this dataset yielded a significant effect favoring MCQs (SMD = - 0.86; 95% CI - 1.01 to - 0.70; p < 0.00001) with markedly reduced heterogeneity (I² = 4%). Meta-analysis of psychometric properties demonstrated strong discrimination (pooled Fisher's z = 1.45) and acceptable reliability (pooled Fisher's z = 0.43) for VSAQs. Evidence on practice effects was mixed, with no consistent advantage for either format.

VSAQs exhibit strong psychometric integrity and are designed to reduce the cueing effect observed in MCQs. Although associated with lower student scores, VSAQs likely provide a more accurate reflection of independent knowledge and clinical reasoning ability. Their integration into medical assessment systems may enhance the authenticity and validity of written examinations.