Related Experiment Video
Updated: Apr 14, 2026
Setting Up a Stroke Team Algorithm and Conducting Simulation-based Training in the Emergency Department - A Practical Guide
Published on: January 15, 2017
Structured performance assessment in three pediatric emergency scenarios: a validation study
Jan Schmutz1, Tanja Manser2, Julia Keil3
1Department of Management, Technology, and Economics, ETH Zurich, Zurich, Switzerland.
Insights
Performance evaluation checklists (PECs) for pediatric emergencies like cardiopulmonary arrest and desaturation are valid and reliable tools for training. The checklist for respiratory syncytial virus (RSV) requires further investigation.
Area of Science:
- Medical Education
- Clinical Skills Assessment
- Simulation-Based Training
Background:
- Systematic performance assessment is crucial for effective medical training.
- Developing validated tools is essential for evaluating clinical skills in emergency scenarios.
Purpose of the Study:
- To develop and validate three performance evaluation checklists (PECs).
- To assess clinical performance in cardiopulmonary arrest, dyspnea with oxygen desaturation, and respiratory syncytial virus (RSV) scenarios.
Main Methods:
- An integrative approach was used to develop three PECs.
- PECs rated 50 simulation training sessions with multiple raters.
- Construct validity and interrater reliability were assessed.
Main Results:
- PECs for desaturation and cardiopulmonary arrest scenarios demonstrated validity and reliability.
- The PEC for RSV scenarios showed limited validity and reliability.
Conclusions:
- PECs are valid and reliable tools for assessing performance in two pediatric emergencies.
- The RSV PEC's limitations may stem from the simulation setting, needing further study.
- Validated performance measures are vital for enhancing training and patient care.
Objective:
To develop and validate 3 performance evaluation checklists (PECs) for systematic performance assessment in 3 clinical scenarios: cardiopulmonary arrest, dyspnea with oxygen desaturation after intubation, and respiratory syncytial virus (RSV).
Study Design:
The 3 PECs were developed using an integrative approach and used to rate 50 training sessions in a simulator environment by different raters. Construct validity was tested by correlating the checklist scores with external constructs (ie, global rating, team experience level, and time to action). Further interrater reliability was tested for all 3 PECs.
Results:
The PECs for the desaturation and cardiopulmonary arrest scenarios were valid and reliable, whereas the PEC for RSV had limited validity and reliability.
Conclusion:
For 2 pediatric emergencies, the PEC is a valid and reliable tool for systematic performance assessment. The unsatisfactory results for the PEC for RSV may be related to limitations of the simulation setting and require further investigation. Structured assessment of clinical performance can augment feedback on technical performance aspects and is essential for training purposes as well as for research. Only reliable and valid performance measures will allow medical educators to accurately evaluate the behavioral effects of training interventions and further enhance the quality of patient care.
