The accuracy of clinical assessments as a measure for teamwork effectiveness

Team training in healthcare is usually evaluated by observers who either score trainees' behaviors, social skills, and cognitive skills during simulation or measure changes in the clinical state of a mannequin. Both methods have shortcomings that limit their usefulness. We propose Brunswik's probabilistic functionalism and the Accuracy Score (AS), a measure emerging from judgment analysis, as elements of a complementary approach that could increase the objectivity of team training evaluation. We report an initial investigation.

Three groups of neonatal clinicians participated in a resuscitation experiment involving three different training interventions. During the experiment, at various phases, the participants were required to assign an Apgar score to a mannequin.

The AS was used to test how accurately the clinicians assigned Apgar scores to the mannequin across different levels of task demand, training content, and training delivery method.

The AS was lower when task demand increased (P < 0.01). The AS was higher after teamwork training than after clinical training (P < 0.05) and better after hands-on teamwork training than after lecture-based teamwork training (P < 0.05).

Because it is simple and objective, the AS may complement existing measures for team training evaluation. Future studies are required in which the AS is tested with a larger number of trainees, in longitudinal experiments, across different training areas, and is compared with previously validated team performance measures.