MM FD ConvFormer multimodal frequency aware deformable CNN transformer network for robust brain tumor classification.

Area of Science:

• Medical Imaging
• Artificial Intelligence
• Computational Biology

Background:

• Accurate brain tumor classification from MRI is crucial for patient outcomes.
• Current models often miss spectral features and tumor heterogeneity due to reliance on single-modal spatial data.
• Limited generalization across datasets is a significant challenge for existing methods.

Purpose of the Study:

• To develop a robust and interpretable multimodal network for brain tumor classification.
• To improve the capture of spectral features and model tumor heterogeneity.
• To enhance cross-dataset generalization capabilities.

Main Methods:

• Proposed MM-FD-ConvFormer, a multimodal frequency-aware deformable CNN-Transformer network.
• Integrated spatial MRI, frequency-domain MRI (Fourier/wavelet transforms), and multi-scale contextual features.
• Employed ConvNeXt V2 backbone, a parallel ConvNeXt branch, and Swin Transformer V2 with a deformable cross-modal attention mechanism.

Main Results:

• MM-FD-ConvFormer consistently outperformed CNN baselines, transformers, and hybrid models in accuracy, macro-F1 score, and AUC.
• Demonstrated robust performance across multiple public and external validation datasets.
• Qualitative analyses confirmed interpretability and effective tumor localization.

Conclusions:

• MM-FD-ConvFormer provides a superior, interpretable, and generalizable solution for automated brain tumor classification.
• The multimodal, frequency-aware approach effectively addresses limitations of single-modal methods.
• The model shows significant potential for real-world clinical applications in neuro-oncology.