Practical Approach for Evaluating Machine Learning Anomaly Detection Algorithms for Epidemic Early Warning Systems

Anomaly detection methods in time series data can play a pivotal role in epidemic surveillance Early Warning Systems (EWS). Statistical and rules-based methods have been traditionally employed in such systems, but are challenged by data dynamics and necessitate expert fine-tuning regularly. On the other hand, machine learning methods can handle complex and multidimensional data better, learn and adapt to changing patterns, and improve their performance. However, practical methodologies for their fitting and evaluation relative to gold standard data for infectious diseases epidemic surveillance are still lacking. In this study, a practical evaluation method was presented using an ensemble technique of four traditional statistical models to build the reference gold standard dataset, and results of validation of two machine learning (LSTM and Isolation Forest) relative to four pathogen data series (COVID19, Hepatitis C, Acinetobacter baumannii and Methicillin-resistant Staphylococcus aureus) was reported with promising results. Lessons learned can be useful in the perspective of adapting ML algorithms to epidemic surveillance EWS.