Related Experiment Videos
Effects of publishing multiple-choice questions on their subsequent item analysis
R C Bandaranayake1, A J Buzzard, V C Marshall
1Royal Australasian College of Surgeons, Melbourne, Victoria.
The Australian and New Zealand Journal of Surgery
|December 1, 1990
Summary
Publication of multiple-choice questions (MCQs) in a book significantly increases their difficulty, especially for high-achieving students. This suggests MCQs become easier after publication, impacting exam performance.
Area of Science:
- Medical Education
- Psychometrics
Background:
- Multiple-choice questions (MCQs) are widely used in assessments.
- The impact of question publication on subsequent examination performance is not fully understood.
Purpose of the Study:
- To investigate the effect of publishing MCQs from a question bank on their performance in later examinations.
- To analyze changes in question difficulty and discrimination indices post-publication.
Main Methods:
- A prospective study comparing 63 published MCQs with 63 unpublished control questions.
- Analysis of facility index and biserial discriminating index.
- Comparison of performance between upper and lower quartiles of candidates.
Main Results:
- Published MCQs showed a significant increase in the mean facility index post-publication (P < 0.01).
- No significant changes were observed in the mean facility index of control questions or the mean discriminating index of any questions.
- The increase in facility index for published questions was greater for abler candidates than for weaker candidates (P < 0.01).
Conclusions:
- Publication of MCQs from a question bank makes them easier to answer.
- Adjusting passing scores based on the proportion of published questions is not supported.
- Less able candidates should be advised to utilize published MCQ books for exam preparation.