Unsupervised learning enables instance-level microservice fault detection using traces and resource metrics.

Area of Science:

• Computer Science
• Software Engineering
• Artificial Intelligence

Background:

• Microservice systems present complex, dynamic dependencies, complicating fault detection.
• Elastic scaling and load balancing obscure fault propagation and diversify anomaly patterns.
• Instance-level fault detection is challenging, particularly without labeled fault data.

Purpose of the Study:

• To develop an unsupervised, likelihood-based framework for instance-level microservice fault detection.
• To address the challenge of detecting faults in complex microservice environments with limited labeled data.
• To leverage distributed traces and resource metrics for robust fault identification.

Main Methods:

• Encoding request traces into Multidimensional Feature Traces (MFTs) in a compact COO format.
• Utilizing edit-distance based prototype clustering to handle heterogeneous trace structures.
• Employing a BiLSTM-VAE with RealNVP flow for modeling normal baselines and capturing sequential dependencies.
• Implementing KDE-based tail-probability hypothesis testing for online detection.

Main Results:

• Achieved high F1-scores of 0.979 on AIOps2020 and 0.985 on TrainTicket datasets.
• Outperformed several strong baselines in instance-level fault detection.
• Demonstrated the significant contributions of multidimensional feature fusion, pattern-wise baseline separation, and posterior flow modeling.

Conclusions:

• The proposed unsupervised framework is effective for instance-level microservice fault detection.
• The method successfully handles complex dependencies and diverse anomaly patterns in microservices.
• The framework offers a robust solution for fault detection in scenarios with unavailable labeled data.