Beyond baselines for anomaly detection: Isolation Forest for baseline-free identification of anomalies in microbial...

Flow cytometry can optimize and complement conventional, growth-based methods for monitoring of the microbial quality of drinking water. Data analysis however largely relies on observing deviations from baselines compiled over time using cell counts and/or nucleic acid content, and more complex shifts in the microbial community are not considered. The multi-parametric data provided by flow cytometry could deliver more nuanced information about microbial water quality, however, as the data are unlabeled, dense and influenced by seasonal, hydrometeorological and operational variations, interpretation has been challenging. Isolation Forest, a computationally inexpensive, unsupervised, multivariate anomaly detection algorithm, was used to interpret 13,951 low-resolution flow cytometric fingerprints describing microbial quality of ground water at two geographically distinct sites. Normalized and non-normalized features captured two different anomaly types and SHAP analysis on the Isolation Forest models identified regions in the fingerprints driving anomalies. As data was collected from operating extraction wells, negating the possibility for ground truth anomaly labels, anomalies assigned were linked to environmental and operational data. This was then indirectly evaluated by predicting anomaly labels using a Random Forest classification model and the environmental and operational data, reaching an accuracy of ∼90 %. Important features matching site-specific hydrological and microbial domain knowledge were then identified by subsequent SHAP analysis on the Random Forest model. Isolation Forest combined with SHAP and explainable AI can thus provide a baseline-free approach to identify and interpret site-specific microbial anomalies in microbial water monitoring, while providing links to domain knowledge to support decisions and operational actions for improved extraction well management.