Refine Then Fusion: Robust 3D Brain MRI Synthesis via Vision-Language Collaboration

Area of Science:

• Medical Imaging
• Artificial Intelligence
• Computer Vision

Background:

• Metadata-guided 3D MRI synthesis improves imaging flexibility but faces limitations in spatial dependency handling and vision-language alignment.
• Existing methods struggle with ambiguous features, blurred anatomical boundaries, and imprecise semantic integration due to conventional cross-attention mechanisms.

Purpose of the Study:

• To propose RTFSyn, a novel framework for metadata-guided 3D MRI synthesis that enhances vision-language collaboration.
• To address limitations in spatial dependency representation and cross-modal alignment in current 3D MRI synthesis techniques.

Main Methods:

• Developed an axis-aware visual refinement module to capture directional dependencies and improve structural representation.
• Introduced a cross-modal adaptive fusion module with pixel packing-recovery for efficient cross-attention and text-conditioned dynamic convolution for fine-grained semantic injection.
• Utilized an implicit neural decoder for flexible, high-fidelity synthesis of target MRI modalities.

Main Results:

• RTFSyn achieved superior quantitative performance compared to state-of-the-art methods across four multi-center datasets.
• Demonstrated robust performance against imaging artifacts, in zero-shot evaluations, and across multi-dimensional clinical validations.
• The framework exhibited favorable computational efficiency, high fidelity, and robustness.

Conclusions:

• RTFSyn effectively integrates spatial refinement and adaptive feature fusion for precise cross-modal alignment in 3D MRI synthesis.
• The proposed method shows significant potential for clinical applications due to its high fidelity, robustness, and efficiency.