Related Experiment Video
Updated: Dec 4, 2025
A Computerized Functional Skills Assessment and Training Program Targeting Technology Based Everyday Functional Skills
Published on: February 13, 2020
Screen-Based Simulation for Training and Automated Assessment of Teamwork Skills: Comparing 2 Modes With Different
Randolph H Steadman1, Yue Ming Huang, Markus R Iseli
1From the Department of Anesthesiology and Critical Care (R.H.S.), Houston Methodist Hospital, Houston, TX; Department of Anesthesiology and Perioperative Medicine (Y.M.H.), David Geffen School of Medicine at UCLA; UCLA Simulation Center (Y.M.H.); National Center for Research on Evaluation, Standards, and Student Testing (CRESST) (M.R.I.); CRESST (J.J.L., A.D.K.), Los Angeles, CA; Department of Surgery (A.T.), David Geffen School of Medicine at UCLA; UCLA Center for Advanced Surgical & Interventional Technology Accredited Education Institute (A.T.), Los Angeles; Consultant, (M.D.D.R.) Claremont, CA; UCLA Graduate School of Education and Information Studies (R.L.); David Geffen School of Medicine at UCLA Dean's Office/UCLA Simulation Center (R.K.); UCLA Graduate School of Education and Information Studies (F.R.); Department of Medicine (F.R.), David Geffen School of Medicine at UCLA, Los Angeles, CA; Department of Culture & Communication (S.M.S.), Linköping Universitet, Linköping, Sweden; Department of Surgery (Y.-Y.J.), David Geffen School of Medicine at UCLA, Los Angeles; Community Memorial Health System (C.R.), Ventura; Department of Anesthesiology and Perioperative Medicine (S.P.P.), David Geffen School of Medicine at UCLA; and UCLA Graduate School of Education and Information Studies (N.M.W.), Los Angeles, CA.
Introduction:
The need for teamwork training is well documented; however, teaching these skills is challenging given the logistics of assembling individual team members together to train in person. We designed 2 modes of screen-based simulation for training teamwork skills to assess whether interactivity with nonplayer characters was necessary for in-game performance gains or for player satisfaction with the experience.
Methods:
Mixed, randomized, repeated measures study with licensed healthcare providers block-stratified and randomized to evaluation-participant observes and evaluates the team player in 3 scenarios-and game play-participant is immersed as the leader in the same 3 scenarios. Teamwork construct scores (leadership, communication, situation monitoring, mutual support) from an ontology-based, Bayesian network assessment model were analyzed using mixed randomized repeated measures analyses of variance to compare performance, across scenarios and modes. Learning was measured by pretest and posttest quiz scores. User experience was evaluated using χ2 analyses.
Results:
Among 166 recruited and randomized participants, 120 enrolled in the study and 109 had complete data for analysis. Mean composite teamwork Bayesian network scores improved for successive scenarios in both modes, with evaluation scores statistically higher than game play for every teamwork construct and scenario (r = 0.73, P = 0.000). Quiz scores improved from pretest to posttest (P = 0.004), but differences between modes were not significant.
Conclusions:
For training teamwork skills using screen-based simulation, interactivity of the player with the nonplayer characters is not necessary for in-game performance gains or for player satisfaction with the experience.
