Variation in passing standards for graduation-level knowledge items at UK medical schools

Area of Science:

• Medical Education
• Assessment and Evaluation
• Higher Education

Background:

• UK medical schools lack a unified standard for student passing criteria in graduation-level examinations.
• Variability in independently set standards can impact the perceived rigor and comparability of medical qualifications.

Purpose of the Study:

• To compare independently set passing standards for common single-best-answer (multiple-choice) items in UK medical school graduation exams.
• To explore potential reasons contributing to observed differences in these standards.

Main Methods:

• A repeated cross-sectional study involving UK medical schools over two academic years (2013-2014 and 2014-2015).
• Participating schools used their own standard-setting processes (Angoff or Ebel methods) for a common set of applied knowledge examination items.
• Analysis focused on comparing the passing standards set for these common items.

Main Results:

• Statistically significant differences in passing standards were observed across participating UK medical schools in both study years.
• The interquartile range of passing standards indicated a notable spread, with differences of 5.7 to 6.5 percentage points.
• Factors such as time per item, standard-setting method, and examination timing did not significantly influence the set standards.

Conclusions:

• Independently determined passing standards for common multiple-choice items in UK medical graduation exams exhibit considerable variation.
• Further in-depth examination of standard-setting procedures is required to elucidate the causes of this variability and to develop strategies for its reduction.