Apache Spark and Deep Learning Models for High-Performance Network Intrusion Detection Using CSE-CIC-IDS2018

Area of Science:

• Computer Science
• Cybersecurity
• Artificial Intelligence

Background:

• Growing networks and new technologies expand cybercriminal attack surfaces.
• Network intrusion detection systems (NIDS) are crucial for protecting business data confidentiality, availability, and integrity.
• The CSE-CIC-IDS2018 dataset offers a comprehensive resource for training intrusion detection models with 14 attack types.

Purpose of the Study:

• To propose and evaluate advanced models for multiclassification network intrusion detection.
• To improve the accuracy and effectiveness of identifying diverse cyber threats.
• To compare the performance of deep learning (CNN, LSTM) and Apache Spark models on a large-scale dataset.

Main Methods:

• Feature selection using Random Forests (RF) reduced 84 features to 19.
• Oversampling and undersampling techniques were applied to address dataset imbalance.
• Three models were developed: Convolutional Neural Networks (CNN), Long Short-Term Memory (LSTM), and Apache Spark.

Main Results:

• The Apache Spark model demonstrated superior performance across all 15 classes.
• Achieved up to 100% accuracy for all classes.
• The Apache Spark model recorded the highest F1-scores, reaching 1.00 for most classes.

Conclusions:

• The proposed models, particularly Apache Spark, show outstanding results for multiclassification network intrusion detection.
• Advanced machine learning techniques significantly enhance the ability to detect a wide range of network attacks.
• This research contributes to more robust and effective cybersecurity solutions.