MATR: Multimodal Medical Image Fusion via Multiscale Adaptive Transformer.

Area of Science:

• Medical Imaging
• Artificial Intelligence
• Biomedical Engineering

Background:

• Medical imaging limitations hinder simultaneous acquisition of functional and structural data.
• Multimodal image fusion merges complementary information for improved diagnosis and navigation.
• Current deep learning fusion methods struggle with preserving global context.

Purpose of the Study:

• To develop a novel unsupervised deep learning method for accurate multimodal medical image fusion.
• To address the limitations of existing methods in capturing global context information.
• To enhance feature representation and information preservation in fused medical images.

Main Methods:

• Proposed a Multiscale Adaptive Transformer (MATR) for unsupervised medical image fusion.
• Introduced adaptive convolution to modulate kernels based on global context.
• Employed an adaptive Transformer to improve long-range dependency modeling and semantic extraction.
• Designed a multiscale network architecture for comprehensive information acquisition.
• Devised an objective function with structural and region mutual information loss for information preservation.

Main Results:

• The proposed MATR method significantly outperforms existing state-of-the-art methods.
• Achieved superior visual quality and quantitative evaluation metrics in experiments.
• Demonstrated good generalization capability by extending to other biomedical image fusion tasks.
• The method effectively preserves both structural and feature-level information.

Conclusions:

• MATR offers an effective solution for multimodal medical image fusion, overcoming limitations of previous deep learning approaches.
• The method's ability to capture both local and global information leads to enhanced diagnostic accuracy.
• MATR shows promise for various biomedical image fusion applications due to its robust performance and generalization.