CLDTLog: System Log Anomaly Detection Method Based on Contrastive Learning and Dual Objective Tasks.

Area of Science:

• Computer Science
• Artificial Intelligence
• Machine Learning

Background:

• System logs are vital for maintainability, enabling troubleshooting and event recording.
• Anomaly detection in system logs is crucial for system health and security.
• Extracting semantic information from unstructured logs is a key challenge in log analysis.

Purpose of the Study:

• To propose CLDTLog, an approach for system log anomaly detection.
• To leverage BERT pre-trained models with contrastive learning and dual-objective tasks.
• To avoid log parsing and its associated uncertainties.

Main Methods:

• Utilized a BERT pre-trained model enhanced with contrastive learning and dual-objective tasks.
• Implemented anomaly detection via a fully connected layer, eliminating the need for log parsing.
• Trained and evaluated the CLDTLog model on HDFS and BGL log datasets.

Main Results:

• Achieved state-of-the-art F1 scores of 0.9971 on HDFS and 0.9999 on BGL.
• Demonstrated excellent generalization performance, achieving an F1 score of 0.9993 on 1% of the BGL dataset.
• Outperformed all existing methods in system log anomaly detection.

Conclusions:

• CLDTLog offers a highly effective and efficient solution for system log anomaly detection.
• The approach eliminates the need for log parsing, simplifying the process and improving accuracy.
• CLDTLog exhibits strong generalization capabilities and significantly reduces training costs.