Multivariate Time Series Anomaly Detection Based on Inverted Transformer with Multivariate Memory Gate

Area of Science:

• Industrial Internet of Things (IoT)
• Time Series Analysis
• Deep Learning

Background:

• Detecting anomalies in industrial IoT multivariate time series is crucial but challenging due to imbalanced data, high dimensionality, and inter-variable disparities.
• Current deep learning methods often fail to capture personalized features and inter-variable dependencies, leading to performance degradation and overfitting on abnormal patterns.

Purpose of the Study:

• To propose an effective deep learning model for multivariate time series anomaly detection in industrial IoT.
• To address the limitations of existing methods in handling imbalanced datasets and capturing complex data dependencies.

Main Methods:

• Introduced ITMMG (Inverted Transformer with Multivariate Memory Gate).
• Employs an inverted token embedding strategy to process multivariate data.
• Utilizes a multivariate memory gate to capture deep dependencies among variables and normal patterns.

Main Results:

• ITMMG demonstrated superior performance in detection accuracy and robustness compared to baseline methods.
• The method effectively captures deep dependencies among variables and individual variable normal patterns.
• Significantly reduced misclassification of anomalous samples during reconstruction.

Conclusions:

• ITMMG offers a robust solution for anomaly detection in industrial IoT multivariate time series.
• The proposed inverted Transformer with multivariate memory gate effectively addresses challenges of imbalanced data and complex dependencies.
• Achieved state-of-the-art performance on standard time series anomaly detection datasets.