Comprehensive Analysis and Evaluation of Anomalous User Activity in Web Server Logs

Area of Science:

• Cybersecurity
• Machine Learning
• Network Security

Background:

• Traditional web server anomaly detection methods struggle with vast datasets and subtle anomalies.
• Analyzing individual user activities is crucial for effective cybersecurity.
• NGINX server logs contain valuable data for identifying suspicious behavior.

Purpose of the Study:

• To develop a novel machine learning framework for web server anomaly detection.
• To enhance the accuracy and efficiency of identifying malicious user activities.
• To integrate algorithmic analysis with expert human evaluation for robust security.

Main Methods:

• Applied the Isolation Forest algorithm to NGINX server logs to detect outlier user behaviors.
• Utilized DBSCAN clustering to categorize anomalies based on request patterns.
• Incorporated post-clustering expert analysis by cybersecurity professionals for validation.

Main Results:

• Successfully identified subtle anomalies and outlier user behaviors often missed by conventional methods.
• Categorized anomalies effectively, distinguishing between benign and potentially harmful activities.
• Enabled targeted security responses like access restrictions and configuration adjustments.

Conclusions:

• The integrated framework significantly advances web server anomaly detection capabilities.
• Combining machine learning with expert insights provides a nuanced approach to cybersecurity.
• A multifaceted strategy is essential for protecting web server infrastructures effectively.