Hybrid noise-resistant technique for malware classification.

Area of Science:

• Cybersecurity
• Machine Learning
• Computer Vision

Background:

• Visualization-based malware detection faces challenges with robustness, cross-dataset validation, and distinguishing similar malware families.
• Visual similarity and perturbations degrade feature extraction, impacting classification performance and generalization.

Purpose of the Study:

• To propose a novel hybrid malware representation framework integrating Convolutional Autoencoder (CAE) and Local Binary Pattern (LBP) for robust malware classification.
• To address limitations in existing frameworks, including robustness, perturbation resilience, scalability, and cross-dataset generalization.

Main Methods:

• Developed a hybrid framework combining CAE-based latent structural learning with LBP-based texture feature extraction.
• Integrated global hierarchical representations from CAE with fine-grained local texture descriptors from LBP.
• Evaluated features using machine learning classifiers, notably XGBoost, and conducted extensive experiments on memory-forensics and BODMAS datasets.

Main Results:

• The proposed CAE+LBP framework achieved high performance with XGBoost (accuracy: 99.90%, precision: 99.79%, recall: 99.92%, F1-score: 99.85%).
• Demonstrated superior performance over standalone feature extraction and conventional CNN models.
• Showcased strong robustness and cross-dataset generalization capabilities across diverse malware and noisy conditions.

Conclusions:

• The hybrid CAE+LBP framework offers a robust and scalable solution for malware classification using visualization.
• The approach effectively distinguishes visually similar malware families and maintains performance under perturbations.
• This study sets a new benchmark for hybrid representation learning in memory-forensics malware visualization.