Using gaming simulation to evaluate bioterrorism and emergency readiness training

The University of Minnesota: Simulations, Exercises and Effective Education: Preparedness and Emergency Response Learning Center uses simulations, which allow trainees to participate in realistic scenarios, to develop and evaluate competency.

In a previous study, participants in Disaster in Franklin County: A Public Health Simulation demonstrated that prior bioterrorism and emergency readiness training (BT/ER) is significantly associated with better performance in a simulated emergency. We conducted a second analysis with a larger data set, remapping simulation questions to the Public Health Preparedness and Response Core Competency Model, Version 1.0.

We performed an outcome evaluation of the impact of public health preparedness training. In particular, we compared individuals with significant BT/ER training to individuals without training on the basis of performance in a simulated emergency. We grouped participants as group 1 (≥45 hours of BT/ER training) and group 2 (<45 hours). Dependent variables included effectiveness of chosen responses within the gaming simulation, which was measured as the proportion of questions answered correctly for each participant. The relationship of effectiveness with significant BT/ER training was estimated using either multiple linear or logistic regression.

For overall effectiveness, group 1 had 2% more correct decisions, on average, than group 2 (P < .001). Group 1 performed significantly better, on average, than group 2 for competency 1.1 (P = .001) and competency 2.3 (P < .001). However, group 1 was significantly worse on competency 1.2 than group 2.

Results indicate that prior training is significantly associated with better performance in a simulated emergency using gaming technology. Effectiveness differed by competency, indicating that more training may be needed in certain competency areas.

Next steps to enhancing the usefulness of simulations in training should go beyond questioning if the learner learned and included questions related to the organizational factors that contributed to simulation effectiveness, and attributes of the simulation that encouraged competency and capacity building.