Comparison of Simulation-based Resuscitation Performance Assessments With In-training Evaluation Reports in Emergency...

Area of Science:

• Medical Education
• Emergency Medicine
• Assessment Methods

Background:

• Competency-based assessment is crucial in postgraduate medical education.
• Simulation-based assessments offer objective evaluation opportunities.
• Comparing simulation with traditional portfolio assessment is needed.

Purpose of the Study:

• To compare emergency medicine (EM) residents' performance in simulation-based OSCEs using the Queen's Simulation Assessment Tool (QSAT) with portfolio assessment via modified in-training evaluation reports (ITERs).
• To understand the inferences drawn from simulation-based OSCE assessment in detail.

Main Methods:

• Prospective observational study involving EM residents from five Canadian academic sites.
• Resuscitation competence evaluated using multicenter simulation-based OSCEs with QSATs and global assessment scores (GAS).
• Portfolio assessment used modified ITERs by CanMEDS roles and GAS; correlational and regression analyses performed.

Main Results:

• Significant positive correlation found between total OSCE and ITER scores (r=0.341).
• Strongest correlations between ITER medical expert score and OSCE GAS, communication, and therapeutic action domains.
• ITER medical expert score predicted OSCE total; OSCE therapeutic action predicted ITER total.

Conclusions:

• Simulation-based OSCEs and ITERs measure different aspects of competence, showing weak to moderate correlations.
• Simulation-based OSCEs using QSAT show promise for assessing medical expert and communicator roles in EM residents.
• A combination of assessment methods provides a more comprehensive evaluation of resident competence.