A Compression-Based Method for Detecting Anomalies in Textual Data.

Area of Science:

• Cybersecurity
• Machine Learning
• Data Science

Background:

• Information and communications technology systems are vital and require robust protection against cyber threats.
• Current security log analysis relies heavily on manual inspection by security analysts.
• Existing methods often require significant configuration and are domain-specific.

Purpose of the Study:

• To propose a parameter-free method for detecting security incidents from structured text data.
• To demonstrate the method's flexibility across various cybersecurity domains.
• To reduce the configuration burden associated with security incident detection.

Main Methods:

• Utilized Normalized Compression Distance (NCD) to extract features from text data.
• Employed Support Vector Machines (SVM) for classification of security events.
• Validated the approach on heterogeneous cybersecurity datasets including HTTP anomalies, spam, Domain Generation Algorithms (DGAs), and sentiment analysis.

Main Results:

• Achieved effective detection of security incidents across multiple domains.
• Demonstrated the method's validity and flexibility in diverse cybersecurity scenarios.
• Showcased a low configuration burden, making the method easily adaptable.

Conclusions:

• The proposed parameter-free method offers a versatile and efficient solution for detecting security incidents.
• NCD combined with SVM provides a powerful, adaptable framework for cybersecurity threat detection.
• This approach simplifies the process of analyzing security logs and identifying potential breaches.