TKRD: Trusted kernel rootkit detection for cybersecurity of VMs based on machine learning and memory forensic...

Area of Science:

• Cybersecurity
• Machine Learning
• Cloud Computing

Background:

• Virtual machines (VMs) in cloud computing are increasingly targeted by malware, including kernel rootkits.
• Memory forensics offers a valuable approach for detecting malicious activities by analyzing memory traces.

Purpose of the Study:

• To propose a novel method (TKRD) for automatically detecting kernel rootkits in private cloud VMs.
• To integrate VM memory forensic analysis with bio-inspired machine learning for enhanced malware detection.

Main Methods:

• Extracting malicious features from VM memory dumps using memory forensic analysis.
• Training and evaluating various machine learning classifiers, including Decision Trees, Rule-based classifiers, Bayesian methods, and Support Vector Machines (SVM).
• Utilizing a bio-inspired machine learning approach for feature analysis and classification.

Main Results:

• The Random Forest classifier demonstrated superior performance among the evaluated models.
• The proposed TKRD method achieved high detection accuracy (0.986) and an excellent area under the receiver operating characteristic curve (AUC) value (0.998).
• The system effectively detected previously unknown kernel rootkits.

Conclusions:

• The combination of VM memory forensics and machine learning, particularly Random Forest, provides an effective solution for kernel rootkit detection.
• The TKRD method offers a robust approach to enhancing cybersecurity in virtualized cloud environments.
• This research contributes to the advancement of automated malware detection techniques in cloud infrastructure.