An Explainable Hybrid CNN-Transformer Architecture for Visual Malware Classification

Area of Science:

• Cybersecurity
• Computer Vision
• Machine Learning

Background:

• Traditional signature-based malware detection struggles with evolving threats.
• Visual malware classification, converting binaries to images, offers a promising alternative.
• Deep learning models like CNNs and Transformers show potential for image-based malware analysis.

Purpose of the Study:

• To develop and evaluate a hybrid deep learning model for enhanced visual malware classification.
• To combine the strengths of Convolutional Neural Networks (CNNs) and Transformers for improved pattern recognition.
• To assess the model's performance, interpretability, and real-time applicability.

Main Methods:

• A hybrid deep learning architecture integrating ConvNeXt-Tiny (CNN) and Swin Transformer.
• Training and validation on benchmark datasets: Malimg, MaleVis, VirusMNIST (61 classes).
• Evaluation using additional datasets (Maldeb, Dumpware-10) and Gradient-weighted Class Activation Mapping (Grad-CAM) for interpretability.

Main Results:

• The hybrid model achieved 94.04% validation accuracy, surpassing individual ConvNeXt-Tiny (92.45%) and Swin Transformer (90.44%) models.
• Achieved high accuracy on extended datasets: 98% on Maldeb and 97% on Dumpware-10.
• Grad-CAM visualization confirmed the complementary feature extraction of CNN and Transformer components.

Conclusions:

• The hybrid deep learning approach provides a robust and interpretable method for visual malware classification.
• The model demonstrates practical applicability through real-time deployment scenarios.
• This research contributes to more effective and trustworthy automated malware detection systems.