SpectFusion: Cross-modal Spectrum-aware Attention Network for Unsupervised Multimodal Medical Image Fusion.

Area of Science:

• Medical imaging
• Artificial Intelligence
• Signal Processing

Background:

• Deep learning, especially Transformer-based methods, excels at medical image fusion by modeling long-range dependencies.
• Existing methods struggle with capturing global information due to local attention mechanisms and often neglect spectral features, limiting fusion performance.

Purpose of the Study:

• To propose SpectFusion, an unsupervised cross-modal spectrum-aware fusion framework for enhanced medical image fusion.
• To address limitations in capturing global information and incorporating spectral features in current deep learning fusion approaches.

Main Methods:

• Developed a spatial-spectrum hybrid block combining gradient retention for local spatial features and Fourier convolution for global frequency features.
• Introduced a cross-modal spectrum-aware attention mechanism for dynamic spatial-spectral information interaction during fusion.
• Incorporated a refined registration module for precise image alignment and defined frequency/spatial domain losses for joint constraint.

Main Results:

• SpectFusion demonstrated superior qualitative and quantitative performance compared to state-of-the-art methods in medical image fusion tasks, including brain tumor imaging.
• The framework adaptively achieves fine-grained fusion by leveraging spatial-spectrum information interactions.
• SpectFusion improved performance in downstream tasks like multimodal medical image segmentation.

Conclusions:

• SpectFusion offers a significant advancement in unsupervised medical image fusion by effectively integrating spatial and spectral information.
• The proposed framework enhances diagnostic accuracy and downstream task performance, showing promise for clinical applications.
• The study highlights the importance of considering both spatial and spectral domains for robust medical image fusion.