Assessment of a clinical performance evaluation tool for use in a simulator-based testing environment: a pilot study

Area of Science:

• Medical Education
• Clinical Skills Assessment
• Simulation-Based Training

Background:

• Objective-structured clinical examinations (OSCEs) are standard for assessing clinical skills.
• Simulator-based testing offers a controlled environment for performance evaluation.
• Validating assessment tools in diverse testing modalities is crucial for reliable evaluation.

Purpose of the Study:

• To assess the clinical performance evaluation tool in a simulator-based testing environment.
• To compare the performance of a standardized scoring system in simulator vs. oral OSCE settings.
• To determine the reliability of a validated oral OSCE scoring system within a simulated clinical context.

Main Methods:

• Twenty-three subjects (students, house officers, chief resident-fellows) were evaluated in standardized simulator encounters.
• Performance was compared against identical oral objective-structured clinical examination (OSCE) control sessions.
• A faculty rater used a validated scoring system (eight skills, rated 1-8; passing = 5.75) for all evaluations.

Main Results:

• Chief resident-fellows achieved "passing" scores in both simulator (mean=6.4) and oral (mean=6.4) exams.
• House officers scored "borderline" [simulator=5.6, oral=5.5], and students scored "failing" [simulator=4.3, oral=4.5].
• Significant differences in mean scores were observed among the three cohorts in both testing arms (p <.01).

Conclusions:

• A standardized oral OSCE scoring system performed equally well in a simulator-based testing environment.
• The clinical performance evaluation tool demonstrated efficacy in assessing trainees across different experience levels.
• Simulator-based assessment using validated scoring systems is a viable method for clinical skills evaluation.