Multimodal Transformer for Accelerated MR Imaging.

Area of Science:

• Medical Imaging
• Artificial Intelligence
• Biomedical Engineering

Background:

• Accelerated multi-modal magnetic resonance (MR) imaging enhances speed and quality by using auxiliary modalities.
• Current methods often lack in-depth fusion mechanisms and rely on Convolutional Neural Networks (CNNs), limiting long-range dependency capture.

Purpose of the Study:

• To propose MTrans, a multi-modal transformer model for improved accelerated MR imaging.
• To address limitations in feature fusion and long-range dependency capture in existing methods.

Main Methods:

• Developed MTrans, a transformer-based framework for multi-modal MR imaging.
• Introduced a novel cross-attention module to integrate multi-scale features between target and auxiliary modalities.
• Utilized transformers for enhanced global information capture compared to CNNs.

Main Results:

• MTrans demonstrated superior performance in MR image reconstruction and super-resolution tasks.
• The cross-attention module effectively leveraged multi-scale information for better target modality enhancement.
• Outperformed state-of-the-art methods on both fastMRI and real-world clinical datasets.

Conclusions:

• MTrans offers a powerful new approach for accelerated multi-modal MR imaging.
• The proposed fusion strategy and transformer architecture significantly improve image quality and reconstruction accuracy.
• MTrans shows great potential for clinical applications requiring fast and high-quality MR imaging.