MM-Net: A MixFormer-Based Multi-Scale Network for Anatomical and Functional Image Fusion

Area of Science:

• Medical Imaging
• Computer Vision
• Artificial Intelligence

Background:

• Multi-modal image fusion is vital in medicine and biology.
• Deep learning (DL) methods dominate multi-modal image fusion.
• Existing DL methods struggle with local/global features and scale diversity.

Purpose of the Study:

• To propose MM-Net, a MixFormer-based multi-scale network for enhanced anatomical and functional image fusion.
• To address limitations in capturing local features, global context, and feature scale diversity.

Main Methods:

• Utilized an improved MixFormer backbone for multi-scale feature extraction.
• Employed a cross-modality feature fusion (CMFF) module with multi-source spatial attention.
• Incorporated multi-scale feature interaction (MSFI) and feature aggregation upsample (FAU) modules.
• Developed a hybrid loss function combining spatial and frequency domain components.

Main Results:

• MM-Net demonstrated superior performance compared to state-of-the-art methods in qualitative and quantitative evaluations.
• The proposed fusion model exhibited robust generalization capabilities across different datasets.
• Experimental results validate the effectiveness of the proposed architecture and fusion strategy.

Conclusions:

• MM-Net successfully addresses the challenges of local/global feature extraction and scale diversity in image fusion.
• The proposed method offers a significant advancement in anatomical and functional image fusion.
• The developed network provides a powerful tool for various medical and biological imaging applications.