[Systematic analysis of evaluations of Objective Structured Clinical Exam (OSCE) multiple circuits: Explanatory...

Area of Science:

• Medical Education
• Clinical Skills Assessment
• Healthcare Professional Training

Context:

• Objective Structured Clinical Examinations (OSCEs) are integral to medical education reforms in France.
• This study examines OSCE score variability and inter-rater reproducibility within this new framework.
• The reform of the 2nd cycle of medical studies (R2C) necessitated the implementation of OSCEs.

Purpose:

• To investigate the variability of student scores in Objective Structured Clinical Examinations (OSCEs).
• To assess the inter-rater reproducibility of OSCE scores.
• To identify factors contributing to score variations in OSCEs.

Summary:

• The station factor predominantly explains OSCE score variability (79.7%).
• Student and circuit factors contribute less significantly to score variations.
• High inter-rater reproducibility (ICC=87.2%) was observed for global OSCE scores.
• Station characteristics and evaluator attributes significantly impact score variations.

Impact:

• Provides insights into the reliability of OSCEs in the French medical education system.
• Highlights the importance of station design and evaluator consistency for standardized assessments.
• Informs strategies for improving the standardization and fairness of OSCE circuits.