Evaluation and comparison of statistical methods for early temporal detection of outbreaks: A simulation-based study

Area of Science:

• Epidemiology
• Biostatistics
• Public Health Surveillance

Background:

• Effective temporal outbreak detection is crucial for public health.
• A wide array of statistical algorithms exist for analyzing surveillance data.
• Systematic evaluation of these algorithms is needed to understand their performance characteristics.

Purpose of the Study:

• To systematically evaluate the performance of 21 statistical algorithms for temporal outbreak detection.
• To identify factors influencing the performance of these detection methods.
• To compare algorithm performance based on metrics like false positive rate and probability of detection.

Main Methods:

• Utilized a large dataset of simulated weekly surveillance time series.
• Assessed 21 statistical algorithms, including 19 from the R package 'surveillance'.
• Calculated performance metrics (e.g., FPR, POD, sensitivity, specificity, PPV, NPV, F1-measure) and used multivariate Poisson regression.

Main Results:

• False positive rates (FPR) ranged from 0.7% to 59.9%; probability of detection (POD) ranged from 43.3% to 88.7%.
• High specificity often correlated with low sensitivity, and vice versa.
• Positive predictive values (PPV) varied widely (6.5%–68.4%), while negative predictive values (NPV) remained high (>94%).
• Time series characteristics, particularly past/current outbreak size and duration, significantly influenced algorithm performance.

Conclusions:

• No single algorithm demonstrated optimal performance across all metrics.
• Algorithm selection for temporal outbreak detection should consider the specific characteristics of the surveillance data and expected outbreak patterns.
• Understanding the trade-offs between sensitivity and specificity is essential for practical application.