Simulation-based assessment to identify critical gaps in safe anesthesia resident performance

Area of Science:

• Medical Education
• Anesthesiology
• Simulation-Based Assessment

Background:

• Valid assessment tools are crucial for identifying anesthesia resident performance gaps.
• Many existing medical assessment tools lack proper validation.
• Expert faculty identified key behavioral domains for evaluating residents.

Purpose of the Study:

• To design and test a behaviorally anchored scale within a simulation system.
• To identify high- and low-performing anesthesia residents.
• To assess critical domains of resident performance relevant to expert faculty.

Main Methods:

• A Delphi process defined five key behavioral domains.
• Seven simulation scenarios assessed 22 residents and 8 fellows.
• Trained faculty used a behaviorally anchored rating scale for blinded scoring.

Main Results:

• The assessment demonstrated strong reliability and validity (generalizability coefficient ρ = 0.81).
• Significant performance differences were observed between residents and fellows.
• Most participants found the assessment useful, realistic, and representative of critical skills.

Conclusions:

• Initial evidence supports the validity of this simulation-based system.
• The system effectively identifies critical gaps in anesthesia resident performance.
• Early identification of performance deficits promotes safer training practices.