Individual and population level uncertainty interact to determine the performance of outbreak surveillance systems

Outbreak detection frequently relies on imperfect individual-level case diagnosis. Both outbreaks and cases are discrete events that can be misclassified and uncertainty at the case level may impact the performance of outbreak alert and detection systems. Here, we describe how the performance of outbreak detection depends on individual-level diagnostic test characteristics and population-level epidemiology, and describe settings where imperfect individual-level tests can achieve consistent performance comparable to "perfect" diagnostic tests.

We generated a stochastic SEIR model to simulate daily incidence of measles (i.e., true) and non-measles (i.e., noise) febrile rash illness. We modeled non-measles sources as either independent static (Poisson) noise, or dynamical noise consistent with an independent SEIR process (e.g., rubella). Defining outbreak alerts as the exceedance of a threshold by the 7-day rolling average of observed test positives, we optimized the threshold that maximized outbreak detection accuracy across sets of noise structures and magnitudes, diagnostic test accuracy (consistent with either a perfect test, or proposed rapid diagnostic tests), and testing rates.

The optimal threshold for each diagnostic test typically increased monotonically with testing rate. With static noise, outbreak detection with RDT-like and perfect tests achieved accuracies of 90%, with comparable delays to outbreak detection. With dynamical noise, the accuracy of perfect test scenarios was superior to those achieved with RDTs (≈ 90% vs. ≤ 80%). Outbreak detection accuracy declined as dynamical noise increased and leads to permanent alert status with RDT-like tests at very high noise. The performance of an outbreak detection system is highly sensitive to the structure and the magnitude of the background noise. Depending on the epidemiological context, outbreak detection using RDTs can perform as well as perfect tests.