Attention-based deep learning models for predicting anomalous shock of wastewater treatment plants

Quickly grasping the time-consuming water quality indicators (WQIs) such as total nitrogen (TN) and total phosphorus (TP) of influent is an essential prerequisite for wastewater treatment plants (WWTPs) to prompt respond to sudden shock loads. Soft detection methods based on machine learning models, especially deep learning models, perform well in predicting the normal fluctuations of these time-consuming WQIs but hardly predict their sudden fluctuations mainly due to the lack of extreme fluctuation data for model training. This work employs attention mechanisms to aid deep learning models in learning patterns of anomalous water quality. The lack of interpretability has always hindered deep learning models from optimizing for different application scenarios. Therefore, the local and global sensitivity analyses are performed based on the best-performing attention-based deep learning and ordinary machine learning models, respectively, allowing for reliable feature importance quantification with a small computational burden. In the case study, three types of attention-based deep learning models were developed, including attention-based multilayer perceptron (A-MLP), Transformer composed of stacked A-MLP encoder and A-MLP decoder, and feature-temporal attention-based long short-term memory (FTA-LSTM) neural network with encoder-decoder architecture. These developed attention-based deep learning models consistently outperform the corresponding baseline models in predicting the testing set of TN, TP, and chemical oxygen demand (COD) time series and the anomalous values therein, clearly demonstrating the positive effect of the integrated attention mechanism. Among them, the prediction performance of FTA-LSTM outperforms A-MLP and Transformer (2.01-38.48 % higher R², 0-85.14 % higher F1-score, 0-62.57 % higher F2-score). Predicting anomalous water quality using attention-based deep learning models is a novel attempt that drives the WWTPs' operation towards being safer, cleaner, and more cost-efficient.