Evading control flow graph based GNN malware detectors via active opcode insertion method with maliciousness...

Area of Science:

• Computer Science
• Cybersecurity
• Machine Learning

Background:

• Machine learning advancements drive new malware detection techniques.
• Existing adversarial attacks struggle to modify portable executable (PE) control flow graphs (CFGs).
• Graph neural network (GNN) models using CFGs pose detection challenges for adversarial malware.

Purpose of the Study:

• To develop a novel method for generating adversarial malware that evades GNN detection.
• To address limitations in current function-preserving adversarial attacks on PE malware CFGs.
• To introduce an autonomous approach for creating effective adversarial malware.

Main Methods:

• Introduced active opcode insertion, a novel base modification technique for PE CFGs.
• Utilized reinforcement learning within the MalAOI framework to identify optimal insertion points and opcode sequences.
• Generated adversarial malware by modifying PE CFGs while preserving functionality.

Main Results:

• MalAOI achieved an average evasion rate of 93.73% against GNN detection models.
• The generated adversarial malware showed only a 12.87% increase in byte size.
• Tested on BODMAS and SOREL-20M datasets, confirming effectiveness.

Conclusions:

• Active opcode insertion is an effective strategy for generating functional adversarial malware.
• MalAOI successfully evades GNN-based malware detection by modifying PE CFGs.
• The method offers a promising solution for bypassing advanced malware detection systems.