A comprehensive study of machine learning techniques for log-based anomaly detection.

Area of Science:

• Computer Science
• Artificial Intelligence
• Software Engineering

Background:

• System complexity necessitates automated log analysis techniques like Log-based Anomaly Detection (LAD).
• Deep learning methods dominate LAD research, but traditional and semi-supervised approaches also warrant consideration.
• Current evaluations of LAD techniques primarily focus on detection accuracy, neglecting crucial practical aspects.

Purpose of the Study:

• To conduct a comprehensive empirical study comparing various supervised and semi-supervised, traditional and deep ML techniques for LAD.
• To evaluate LAD techniques based on detection accuracy, time performance, and sensitivity to hyperparameter tuning.
• To provide robust evidence on the relative strengths and weaknesses of different ML approaches for LAD.

Main Methods:

• Evaluation of a wide array of supervised and semi-supervised, traditional and deep ML techniques.
• Assessment across four criteria: detection accuracy, training time, prediction time, and sensitivity to hyperparameter tuning.
• Empirical comparison using benchmark datasets relevant to LAD.

Main Results:

• Supervised traditional and deep ML techniques exhibit comparable detection accuracy and prediction times on most benchmark datasets.
• Traditional ML techniques demonstrate lower sensitivity to hyperparameter tuning compared to deep learning methods.
• Semi-supervised techniques generally achieve significantly lower detection accuracy than supervised methods.

Conclusions:

• Both traditional and deep supervised ML techniques are viable for LAD, with traditional methods offering better hyperparameter robustness.
• The choice of LAD technique should consider not only accuracy but also computational performance and sensitivity to tuning.
• Further research should explore the practical implications of these findings for system engineers.