Related Experiment Video
Updated: Nov 6, 2025
Emergency Undocking in Robotic Surgery: A Simulation Curriculum
Published on: May 20, 2018
Tools for Assessing the Performance of Pediatric Perioperative Teams During Simulated Crises: A Psychometric Analysis
Scott C Watkins1, Getulio R de Oliveira Filho, Cory M Furse
1From the Johns Hopkins All Children's Hospital (S.C.W.)St. Petersburg, FL; Federal University of Santa Catarina (G.R.d.O.F.), Florianópolis, Brazil; Medical University of South Carolina (C.M.F., A.T.R.), Charleston, SC; Stanford University Medical Center (M.K.M., R.J.R., B.M.), Palo Alto, CA; and Vanderbilt University Medical Center (M.D.M.), Nashville, TN.
Introduction:
The pediatric perioperative setting is a dynamic clinical environment where multidisciplinary interprofessional teams interact to deliver complex care to patients. This environment requires clinical teams to possess high levels of complex technical and nontechnical skills. For perioperative teams to identify and maintain clinical competency, well-developed and easy-to-use measures of competency are needed.
Methods:
Tools for measuring the technical and nontechnical performance of perioperative teams were developed and/or identified, and a group of raters were trained to use the instruments. The trained raters used the tools to assess pediatric teams managing simulated emergencies. A psychometric analysis of the trained raters' scores using the different instruments was performed and the agreement between the trained raters' scores and a reference score was determined.
Results:
Five raters were trained and scored 96 recordings of perioperative teams managing simulated emergencies. Scores from both technical skills assessment tools demonstrated significant reliability within and between ratings with the scenario-specific performance checklist tool demonstrating greater interrater agreement than scores from the global rating scale. Scores from both technical skills assessment tools correlated well with the other and with the reference standard scores. Scores from the Team Emergency Assessment Measure nontechnical assessment tool were more reliable within and between raters and correlated better with the reference standard than scores from the BARS tool.
Conclusions:
The clinicians trained in this study were able to use the technical performance assessment tools with reliable results that correlated well with reference scores. There was more variability between the raters' scores and less correlation with the reference standard when the raters used the nontechnical assessment tools. The global rating scale used in this study was able to measure the performance of teams across a variety of scenarios and may be generalizable for assessing teams in other clinical scenarios. The Team Emergency Assessment Measure tool demonstrated reliable measures when used to assess interprofessional perioperative teams in this study.
