Hybrid Multi-View MRI Fusion for csPCa Diagnosis via Intra- and Inter-View Transformers

Area of Science:

• Medical Imaging
• Artificial Intelligence
• Oncology

Background:

• Accurate diagnosis of clinically significant prostate cancer (csPCa) is crucial for treatment.
• Current deep learning methods often lose spatial information due to late fusion strategies.
• Existing Vision Transformers have high memory demands, and Swin Transformers are limited to single views.

Purpose of the Study:

• To develop a novel hybrid fusion framework for improved csPCa diagnosis from multi-view MRI.
• To overcome limitations of late fusion, Vision Transformers, and Swin Transformers.
• To enhance spatial coherence and fine-grained feature integration for better diagnostic performance.

Main Methods:

• Proposed a hybrid fusion framework with iterative intra-view and inter-view interactions across resolutions.
• Utilized a Vision Transformer-based inter-view module with bridge tokens for efficient cross-view information summarization.
• Employed a Swin Transformer-based intra-view module for dynamic patch and token interactions within windows.
• Incorporated shared positional embeddings to maintain spatial correspondence across MRI views (axial, sagittal, coronal).

Main Results:

• The proposed hybrid framework significantly outperformed existing methods in csPCa classification on a public dataset.
• Ablation studies confirmed the contribution of individual framework components.
• Attention map visualizations demonstrated effective integration of anatomical structures across different MRI views.

Conclusions:

• The novel hybrid fusion framework effectively integrates multi-view MRI data for accurate csPCa diagnosis.
• The method preserves spatial information and anatomical correspondence while maintaining computational efficiency.
• This approach offers a promising advancement for prostate cancer detection and treatment planning.