Intelligent model for the detection and classification of encrypted network traffic in cloud infrastructure

Area of Science:

• Cybersecurity
• Network Security
• Machine Learning Applications

Background:

• Domain Name Server (DNS) protocol is vulnerable to security flaws and cyber-attacks.
• DNS abuse is a significant cybersecurity concern, despite advancements like HTTPS.
• DNS over HTTPS (DoH) was developed to enhance DNS security and privacy against eavesdropping and tampering.

Purpose of the Study:

• To explore machine learning (ML) methods for detecting and categorizing network traffic data, focusing on the DNS protocol.
• To evaluate the effectiveness of different ML models in classifying malicious versus non-DoH traffic.

Main Methods:

• Utilized a machine learning-based classification approach on a network traffic dataset.
• Employed AdaBoost, Support Vector Classification with Radial Basis Function (SVC-RBF), and Quadratic Discriminant Analysis (QDA) models.

Main Results:

• AdaBoost model achieved 75% accuracy for malicious and 73% for non-DoH traffic classification.
• SVC-RBF model attained 76% accuracy in classifying malicious versus non-DoH traffic.
• QDA model demonstrated high performance, with 99% accuracy for malicious and 98% for non-DoH traffic classification.

Conclusions:

• Machine learning models, particularly QDA, show significant promise in detecting and categorizing malicious DNS traffic.
• Implementing advanced ML techniques can enhance the security of DNS communications, including DNS over HTTPS (DoH).
• The study highlights the effectiveness of ML in addressing evolving cybersecurity threats in network protocols.