Detecting Reconnaissance and Discovery Tactics from the MITRE ATT&CK Framework in Zeek Conn Logs Using Spark's...

Area of Science:

• Computer Science
• Cybersecurity
• Data Science

Background:

• The increasing complexity and volume of network traffic necessitate advanced intrusion detection systems (IDS).
• Existing intrusion detection datasets may lack relevance to current cyber threats, impacting the effectiveness of machine learning models.
• Scalability and real-time performance are critical challenges for modern IDS.

Purpose of the Study:

• To introduce and utilize the novel UWF-ZeekData22 dataset for network intrusion detection research.
• To evaluate the performance, scalability, and response time of various machine learning classifiers on this new dataset.
• To analyze the effectiveness of these classifiers in detecting reconnaissance and discovery tactics within network data.

Main Methods:

• Development and utilization of the UWF-ZeekData22 dataset, comprising Zeek Connection Logs from Security Onion 2.
• Labeling of the dataset using the MITRE ATT&CK framework's Tactics, Techniques, and Procedures (TTPs).
• Application of multiple machine learning classifiers (naïve Bayes, random forest, decision tree, support vector classifier, gradient boosted trees, logistic regression) using the Spark big data framework.

Main Results:

• Performance evaluation of several machine learning classifiers on the UWF-ZeekData22 dataset.
• Analysis of the scalability and response times of these classifiers when processed using Spark.
• Identification of effective classifiers for detecting specific network intrusion tactics like reconnaissance and discovery.

Conclusions:

• The UWF-ZeekData22 dataset provides a relevant and contemporary resource for IDS research.
• Machine learning classifiers, when implemented with big data frameworks like Spark, show promise for scalable network intrusion detection.
• Further research can leverage this dataset to develop more robust and efficient intrusion detection systems.