Conservative Novelty Synthesizing Network for Malware Recognition in an Open-Set Scenario

Area of Science:

• Cybersecurity
• Machine Learning
• Computer Science

Background:

• Traditional malware recognition often assumes known families (closed-set scenario), failing to address novel, emerging threats.
• Real-world cybersecurity requires recognizing malware in open-set scenarios, where unknown families are present, a challenge inadequately addressed.
• Existing classifiers struggle with open-set malware recognition due to overly confident predictions, degrading performance on unknown families.

Purpose of the Study:

• To develop a robust system for malware open-set recognition (MOSR) capable of identifying both known and novel malware families.
• To overcome the limitations of conventional classifiers that exhibit degraded performance on unknown malware instances.
• To propose a novel model that enhances the classification and detection of unknown malware families.

Main Methods:

• Proposed a novel model utilizing generative adversarial networks (GANs) to synthesize marginal malware instances mimicking unknown families.
• Employed a conservative synthesis approach to guide classifiers, lowering probabilities for unknown families and raising them for known ones.
• Implemented a cooperative training scheme integrating classification, synthesis, and rectification for improved model performance.

Main Results:

• The proposed model demonstrated superior performance in malware open-set recognition compared to existing methods.
• Synthesized malware instances effectively trained the classifier to better distinguish between known and unknown families.
• Experimental results on benchmark datasets and the new MAL-100 dataset validated the model's effectiveness.

Conclusions:

• The novel GAN-based approach significantly improves malware open-set recognition accuracy.
• The developed model offers a practical solution for identifying novel malware families in real-world cybersecurity.
• The creation of the MAL-100 dataset provides a valuable resource for future research in open-set malware recognition.