Comparison of Standard Setting Procedures to Establish Defensible Passing Standards for Clinical Skills Assessment:...

Standard setting methods for clinical skills assessments help establish cut scores that accurately reflect clinical performance expectations. However, these methods lead to varied cut scores, and guidance for method selection is limited. This study compares the application of four methods.

The Angoff, Patient Safety, Borderline Group and Contrasting Groups methods were applied to an assessment of physical therapist student clinical skills. The resulting cut scores were applied to a de-identified historical dataset (n = 92). Post hoc logistic regression analysis evaluated the underlying constructs for global ratings of student competence.

The cut scores ranged from 65.9% to 86.1%, with the Angoff method resulting in the lowest cut score and the Borderline Group method resulting in the highest cut score. Applying the cut scores to the historical dataset resulted in pass rates ranging from 60.9% to 98.9%. Logistic regression modelling revealed that increasing Safety Score was associated with an increased likelihood of receiving a 'pass' global rating with an odds ratio of 18.97 (95% CI, 2.30-156.63). Total Score did not have a statistically significant association.

The Patient Safety method required a higher performance level for items related to safety, aligned with expert judges' conceptualisation of competence. Therefore, the Patient Safety method demonstrated the best match for this assessment's goals. These findings can inform method selection for clinical skills where patient safety is a key consideration. Additional recommendations are to include pass rate feasibility and conceptual alignment with the target construct when selecting standard setting methods.