How can the examination failure rate be stabilised?

Area of Science:

• Medical Education
• Assessment Methods
• Psychometrics

Background:

• Medical examinations at NTNU have a fixed pass mark of 65% correct answers.
• Annual variations in examination difficulty lead to fluctuating failure rates.
• The study hypothesized that varying question difficulty causes failure rate fluctuations.

Purpose of the Study:

• To investigate if relative standard-setting methods reduce failure rate variation.
• To determine if these methods maintain the pass grade limit.
• To compare Cohen's methods with the existing absolute pass mark.

Main Methods:

• Utilized data from 34 medical examinations at NTNU (2010-2015).
• Applied Cohen's relative standard-setting methods.
• Compared failure rates from Cohen's methods against the existing 65% absolute pass mark.

Main Results:

• The existing 65% pass mark resulted in failure rates from 0% to 13.7%.
• Cohen's methods generally produced lower failure rates with less variation compared to the existing method.
• One Cohen method yielded failure rates between 0% and 10.4%.

Conclusions:

• An absolute 65% pass mark is defensible given generally low failure rates in this study.
• Cohen's methods are a viable alternative for medical schools with high or highly variable failure rates.
• Relative standard-setting can enhance assessment consistency in medical programs.