Related Experiment Video
Updated: Mar 1, 2026
08:08
Simulator Training for Endovascular Neurosurgery
Published on: May 6, 2020
4.2K
Simulation-Based Assessment Identifies Longitudinal Changes in Cognitive Skills in an Anesthesiology Residency
Avner Sidi1, Nikolaus Gravenstein, Terrie Vasilopoulos1
1From the Department of Anesthesiology.
Journal of Patient Safety
|June 6, 2017
Summary
Simulation-based assessments effectively track anesthesia residents' cognitive skills, revealing improvements in higher-order deficiencies over time. This method aids in identifying and remediating cognitive performance gaps not caught by traditional tools.
Area of Science:
- Medical Education
- Anesthesiology Training
- Cognitive Skill Assessment
Background:
- Traditional assessment tools often fail to identify nontechnical or "higher-order" cognitive deficiencies in medical residents.
- Anesthesia residency programs require robust methods to evaluate and improve residents' cognitive performance and nontechnical skills.
- Simulation offers a promising avenue for assessing complex cognitive skills beyond basic technical knowledge.
Purpose of the Study:
- To evaluate higher-order cognitive performance in anesthesia residents.
- To demonstrate the efficacy of simulation as an assessment tool for cognitive skills and "higher-order" deficiencies.
- To hypothesize that simulation can identify longitudinal changes in cognitive skills and facilitate remediation.
Main Methods:
- Fifty simulation scenarios were used to evaluate 35 anesthesia residents over two academic years.
- Residents were assessed using scenario-specific checklists evaluating basic knowledge/technical (lower-order) and advanced cognitive/nontechnical (higher-order) items.
- A subset of 18 residents repeated identical or similar scenarios annually, with error and success rates calculated for group and individual performance.
Main Results:
- Postgraduate Year 4 (PGY4) residents demonstrated lower error rates and higher success rates in cognitive items compared to technical skills.
- Cognitive error rates for PGY4 residents were low (0.00-0.22) and success rates high (0.83-1.00) across three clinical domains, showing significant improvement.
- Annual decreases in error rates were observed over two years, primarily driven by reductions in cognitive errors, with common biases including anchoring and confirmation bias.
Conclusions:
- Simulation-based assessments effectively highlight residents' cognitive strengths, weaknesses, and progress over time.
- These assessments can inform curriculum development by identifying needs for activities focusing on higher-level cognitive processing.
- Simulation provides a valuable tool for monitoring and improving cognitive performance in anesthesia residency training.
