Manod: A multi-modal anomaly detection framework for distributed system

Area of Science:

• Computer Science
• Software Engineering
• Systems Engineering

Background:

• Distributed infrastructure is crucial for scalable applications, demanding robust anomaly detection for system stability.
• Current anomaly detection methods often use single data sources (metrics or logs), leading to inaccuracies and false positives.
• Integrating multimodal data (metrics and logs) offers a more comprehensive system view for improved fault detection.

Purpose of the Study:

• To propose Manod, a novel semi-supervised fault detection method for monitoring distributed system health.
• To leverage multimodal data (metrics and logs) for more accurate and reliable anomaly identification.
• To improve the accuracy and reduce false positives/negatives in detecting system abnormalities.

Main Methods:

• Employs a graph-based hierarchical encoding approach to generate discriminative representations.
• Utilizes pre-trained language models to process and model both system metrics and logs.
• Introduces a gated attention fusion mechanism for effective integration of heterogeneous data modalities.

Main Results:

• Manod achieved high F1-scores of 0.870 on a simulation dataset (D1) and 0.934 on a real-world dataset (D2).
• The proposed method significantly outperformed all baseline models in fault detection tasks.
• Demonstrated effectiveness in mitigating both false positives and false negatives in anomaly detection.

Conclusions:

• Manod provides an effective semi-supervised approach for fault detection in distributed systems using multimodal data.
• The integration of metrics and logs via advanced fusion techniques enhances system health monitoring capabilities.
• The method shows strong potential for ensuring the stable and reliable operation of large-scale software systems.