Cohen's Method and Its Variations in Standard Setting for Medical Examinations: An Australian Multi-Institutional...

Standard setting is essential for assessing medical students to ensure they meet required competencies for clinical practice. Various methods exist, but no gold standard has been universally adopted. This study aimed to identify the methods of standard setting used in Australian medical schools, and their benefits, limitations and challenges with implementation, to inform a proposal to change standard-setting methods at James Cook University.

A survey designed to gather detailed information on current standard-setting methods, challenges and satisfaction levels related to exit-level written assessments was sent to the assessment leads of all 21 Australian medical schools with graduating students. Survey data were analysed using descriptive statistics and thematic analysis.

All 21 schools participated. Standard-setting methods included Cohen/Modified Cohen (n = 10), Angoff (n = 4), Ebel (n = 4), Hofstee (n = 2) and Objective Borderline method (n = 1). Satisfaction with the most popular method, Cohen/Modified Cohen, was high with a mean of 83% (range 69%-100%), and three additional schools indicated a plan to implement this method in coming years. Common challenges of standard setting in schools using methods other than Cohen/Modified Cohen included panel recruitment, resource management and maintaining assessment quality. Thematic analysis showed stakeholder involvement and the need for additional resources as necessities in implementing effective standard-setting methods.

Cohen method of standard setting is most commonly used for exit-level written examinations at Australian medical schools and is preferred for its efficiency and adaptability. Transitioning to Cohen/Modified Cohen could align JCU practices with those of most other medical schools in Australia.