MHWT: Wide-range attention modeling using window transformer for multi-modal MRI reconstruction

Area of Science:

• Medical Imaging
• Artificial Intelligence
• Computer Vision

Background:

• The Swin Transformer excels at feature modeling but faces limitations with high-resolution images and long-range dependencies in magnetic resonance image reconstruction due to its fixed window size.
• Capturing both local and global dependencies is crucial for accurate magnetic resonance image reconstruction.

Purpose of the Study:

• To propose a novel multi-modal hybrid window attention Transformer (MHWT) to address the limitations of existing Transformers in high-resolution magnetic resonance image reconstruction.
• To enhance the model's ability to capture both local and global dependencies for superior reconstruction performance.

Main Methods:

• Introduced a retractable attention mechanism and a shape-alternating window design within the Transformer architecture.
• Implemented a variable and shifted window attention strategy for flexible modeling of local and global dependencies.
• Optimized the Transformer encoder with adjustments to normalization and attention score computation for improved training stability.

Main Results:

• The proposed MHWT method demonstrated superior performance compared to state-of-the-art approaches in magnetic resonance image reconstruction across multiple public datasets.
• Achieved significant improvements in both single-modal and multi-modal reconstruction scenarios.
• Exhibited enhanced image reconstruction ability and adaptability, validating the effectiveness of the novel attention mechanisms.

Conclusions:

• The MHWT model effectively overcomes the limitations of fixed window sizes in Transformers for high-resolution magnetic resonance image reconstruction.
• The novel attention mechanisms and architectural improvements lead to state-of-the-art performance and greater adaptability in medical image reconstruction tasks.
• The publicly available code facilitates further research and application of this advanced reconstruction technique.