Measuring the effectiveness of performance-based training

This article describes a statistical analysis of small water systems' turbidity data within the framework of a logic model for the USEPA's Performance-Based Training (PBT) program. The logic model shows the theoretical linkages between optimization training for small system operators; operator application of optimization techniques; improvements in plant filtration performance; and public health protection. The analysis comprised two phases. For the first phase, the authors used Bayesian analysis of turbidity data to test the statistical significance of changes in finished water quality resulting from training for small water system operators. For the second phase, the authors estimated the potential health benefits resulting from measured improvements in filtration performance. Considering only the improved removal of the pathogen Cryptosporidium, the expected annual health benefit of PBT is about ten fewer cases of infection per thousand persons served (within a 95% credible interval 0 to 18 fewer infections), though there may be benefits associated with the removal of other pathogens. The article also describes factors contributing to uncertainty in the estimated potential health benefits. The proposed two-phase approach supports the USEPA's development of drinking water program indicators which are meaningful, measurable, broadly applicable and change-sensitive.