Sequential testing in a high stakes OSCE: Determining number of screening tests

Area of Science:

• Medical Education
• Assessment Methods
• Clinical Skills Evaluation

Background:

• The sequential objective structured clinical exam (OSCE) is a modified assessment format.
• This approach involves an initial screening test, with only underperforming students proceeding to further evaluation.
• Optimizing the number of screening stations and pass marks is crucial for efficiency and accuracy.

Purpose of the Study:

• To evaluate the outcomes of adopting a sequential OSCE approach.
• To determine the optimal number of screening stations and pass marks.
• To analyze the sensitivity and specificity of different sequential OSCE configurations.

Main Methods:

• Retrospective observational study of 388 medical students (2013-2014).
• Used simulation to compare outcomes across 6-14 OSCE stations versus 15 stations.
• Analyzed varying numbers of screening stations and pass marks (±1, +2, +3 SEM).

Main Results:

• Sensitivity ranged from 84-98% across 6-14 stations.
• Specificity varied from 60-100%, generally increasing with more stations.
• Eight stations demonstrated a balance of high sensitivity (88-89%) and specificity (83-86%).

Conclusions:

• The study provides robust data on optimizing sequential OSCE design.
• Identifies eight stations as a potentially ideal screening test configuration.
• Offers recommendations for implementing sequential OSCEs in medical education, considering educational and resource implications.