Related Experiment Video
Updated: Oct 20, 2025
09:52
Setting Up a Stroke Team Algorithm and Conducting Simulation-based Training in the Emergency Department - A Practical Guide
Published on: January 15, 2017
17.4K
Assessment of Emergency Medicine Resident Performance in a Pediatric In Situ Simulation Using Multi-Source Feedback
Michael C Nguyen1, Nicole C Elliott1, Diane P Begany2
1Department of Emergency and Hospital Medicine, Lehigh Valley Hospital and Health Network/University of South Florida Morsani College of Medicine, Allentown, USA.
Cureus
|September 15, 2021
Summary
Multi-source feedback (MSF) using the Queen's Simulation Assessment Tool (QSAT) showed fair agreement in pediatric emergency department simulations. Faculty inclusion is crucial, while self-evaluation offers limited value for this assessment method.
Area of Science:
- Medical Education
- Simulation-Based Assessment
- Emergency Medicine
Background:
- Multi-source feedback (MSF) is mandated by the Accreditation Council for Graduate Medical Education (ACGME).
- The Queen's Simulation Assessment Tool (QSAT) is validated for distinguishing resident performance and shows excellent MSF agreement in adult simulations.
- Previous studies demonstrated QSAT's efficacy in adult simulation settings.
Purpose of the Study:
- To determine the degree of agreement of MSF using the QSAT in a pediatric (Peds) simulation case.
- To evaluate MSF agreement in an in situ pediatric emergency department (ED) setting.
- To assess the impact of different feedback sources on agreement metrics.
Main Methods:
- A pediatric resuscitation case was developed with behavioral anchors on the QSAT (1-5 scale across five categories).
- Data collected from six participants per simulation, including self-evaluation and feedback from peer residents, nurses, and faculty.
- Agreement calculated using intraclass correlation coefficients (ICC).
Main Results:
- Fair agreement (ICC) was observed across 35 simulations with 106 participants.
- No statistically significant difference in agreement was found between different MSF sources.
- Removing self-evaluation yielded the highest ICC; non-faculty MSF agreement was poor.
- QSAT agreement was lower than previously reported for adult simulations.
Conclusions:
- Faculty inclusion is essential for effective MSF using the QSAT in pediatric emergency medicine.
- Self-evaluation has limited utility within the QSAT framework.
- Lower MSF agreement in this study may be attributed to the pediatric focus or in situ simulation environment.
