Related Experiment Videos
Global-local feature fusion: a robust hybrid deep learning model for multiclass brain tumor classification with
Kirti Pant1, Pijush Kanti Dutta Pramanik2,3, Shahid Mohammad Ganie4
1Department of Computer Science and Engineering, Bipin Tripathi Kumaon Institute of Technology, Dwarahat, Uttarakhand, India.
Background:
Accurate classification of brain tumors in MRI scans is critical for effective clinical decision making, yet manual assessment is labor-intensive and subject to variability. Although deep learning models, particularly CNNs and Transformers, have shown potential, each architecture alone has limitations: CNNs excel at local feature extraction but lack global context awareness, while Transformers capture global relationships but may overlook fine-grained details.
Objectives:
To develop a hybrid Transfer Learning-Transformer model that integrates CNN-driven local feature modeling with Transformer-based global reasoning to enhance multi-class brain tumor classification.
Methods:
The proposed workflow comprises three stages: (1) benchmarking standalone CNN (ResNet50, VGG19, ConvNeXtBase, EfficientNetV2B0) and Transformer models (ViT, Swin, DeiT, PoolFormer); (2) constructing two intra-family hybrids-HTL (ResNet50 + ConvNeXtBase) and HTF (PoolFormer + ViT); and (3) fusing them into a final hybrid model (HF). Two public MRI datasets (Figshare, Kaggle) were used. Models were trained, validated, and tested on the Kaggle dataset, while the Figshare dataset was used exclusively for external validation. Performance was assessed using accuracy, precision, recall, F1-score, AUC, Friedman's aligned-rank test, Kendall's W, Holm post hoc analysis, TOPSIS-based ranking, calibration analysis (Brier score), and Grad-CAM++ for interpretability.
Results:
The HF model achieved near-perfect classification (∼99.4% on Kaggle and up to 100% on Figshare), significantly outperforming all baseline and single-architecture models. Statistical and multi-criteria analyses consistently ranked HF as the top-performing model, while calibration results confirmed reliable probability estimation. Grad-CAM++ further indicated tumor-focused decision-making.
Conclusion:
The proposed hybrid model delivers high accuracy, strong generalizability, and reliable, interpretable predictions across MRI datasets, positioning it as a promising solution for AI-assisted brain tumor diagnosis.