Setting defensible performance standards on OSCEs and standardized patient examinations

Area of Science:

• Medical Education
• Assessment and Evaluation
• Health Professions Education

Background:

• Standardized patient assessments and objective structured clinical examinations are vital for high-stakes medical certification and licensure.
• Ensuring standardized, accurate, reliable, equitable, and defensible competence decisions requires valid standard-setting methods (cut-scores).
• Limited research exists on standard-setting methods for performance-based assessments compared to multiple-choice examinations.

Purpose of the Study:

• To provide guidance on establishing defensible standards for performance assessments.
• To focus specifically on performance assessments utilizing standardized patients in simulated medical encounters.
• To discuss and contrast various standard-setting methods, highlighting their strengths and weaknesses.

Main Methods:

• Review and discussion of existing literature on standard-setting methodologies for performance-based assessments.
• Contrast of different standard-setting approaches, analyzing their applicability and limitations.
• Exploration of potential future research directions and enhancements for current protocols.

Main Results:

• Identified a need for more research into standard-setting methods for performance-based assessments.
• Presented various methods for setting standards in performance assessments, discussing their pros and cons.
• Proposed future research avenues and augmentations to existing standard-setting protocols.

Conclusions:

• Defensible standard-setting is critical for the validity and fairness of high-stakes medical assessments.
• Further research is needed to refine and develop appropriate standard-setting methods for performance-based evaluations.
• The article offers a foundation for understanding and improving standard-setting practices in medical education.