Three-Option Multiple-Choice Questions Retain Discrimination and Reliability: A Prospective Randomized Crossover...

To prospectively compare the psychometric performance of 3-option versus 4-option multiple-choice question (MCQ) examinations and evaluate differences in item difficulty, item discrimination, examination reliability, examination performance, and elapsed testing time.

This randomized crossover study was conducted in two required Doctor of Pharmacy courses at a single school of pharmacy, including a first-year pharmacokinetics course and a third-year pharmacotherapy course. Students were assigned to alternating sequences of 3-option and 4-option MCQ examinations throughout the study period. Four-option examinations were developed first, after which one distractor was removed from each item to create the corresponding 3-option version. Item-level psychometric measures, examination-level outcomes, and elapsed testing time were compared between formats using paired analyses and linear mixed-effects modeling.

Three-option MCQs were associated with students answering approximately one additional question correctly on a 50-item MCQ test compared with four-option MCQs. No differences were observed in point-biserial discrimination or KR-20 reliability estimates between examination formats. Course-specific analyses demonstrated differences in item difficulty within pharmacokinetics examinations but not pharmacotherapy examinations. Examinations containing 4-option MCQs also required longer testing times for students.

Reducing MCQs from four to three answer options did not meaningfully compromise examination reliability or item discrimination in two required pharmacy courses. Although 4-option MCQs produced slightly greater difficulty and longer testing times, the practical educational benefit of routinely requiring a fourth answer option may be limited when distractors are not consistently plausible or functioning effectively.