Multi-Scale Transformer Network With Edge-Aware Pre-Training for Cross-Modality MR Image Synthesis.

Area of Science:

• Medical Imaging
• Computer Vision
• Machine Learning

Background:

• Cross-modality magnetic resonance (MR) image synthesis aims to generate missing MR sequences from existing ones.
• Supervised learning methods typically require large amounts of paired multi-modal data, which are often difficult to acquire.
• Existing challenges include the scarcity of paired data and the need for effective utilization of both paired and unpaired datasets.

Purpose of the Study:

• To propose a novel Multi-scale Transformer Network (MT-Net) for cross-modality MR image synthesis.
• To leverage both limited paired and abundant unpaired data for improved synthesis performance.
• To develop an effective pre-training strategy for enhancing the synthesis model.

Main Methods:

• An Edge-preserving Masked AutoEncoder (Edge-MAE) was pre-trained in a self-supervised manner for image imputation and edge map estimation.
• A novel patch-wise loss function was introduced to improve Edge-MAE performance by adaptively handling masked patches.
• A Dual-scale Selective Fusion (DSF) module within MT-Net integrates multi-scale features from the pre-trained encoder for synthesis.

Main Results:

• The proposed MT-Net achieved comparable performance to existing methods.
• The model demonstrated effectiveness even when utilizing only 70% of the available paired data.
• The edge-aware pre-training strategy significantly contributed to learning contextual and structural information.

Conclusions:

• MT-Net offers a robust solution for cross-modality MR image synthesis, particularly when paired data is limited.
• The combination of self-supervised pre-training and a multi-scale transformer architecture is effective.
• The developed approach facilitates efficient utilization of available multi-modal MR data.