Enhancing pix2pix with Swin Transformer for Cross Modal Brain CT-MR synthesis.

Area of Science:

• Medical Imaging
• Artificial Intelligence
• Neuroscience

Background:

• Cross-modal medical image synthesis is vital for diagnosing cerebral diseases.
• Conventional models like pix2pix struggle with long-range dependencies and global context, limiting image fidelity.
• Transformer-based architectures offer potential for improved feature capture in medical imaging.

Purpose of the Study:

• To develop an enhanced image-to-image translation framework for high-fidelity cross-modal medical image synthesis.
• To leverage the SwinUNETR transformer architecture for improved capture of local and global anatomical features.
• To evaluate the framework's performance in CT-to-MR and MR-to-CT synthesis for neurological applications.

Main Methods:

• Replaced the U-Net generator in pix2pix with the SwinUNETR transformer architecture.
• Utilized hierarchical self-attention mechanisms for capturing both local and global image features.
• Evaluated the framework on 2,091 paired CT and T1-weighted MR scans from public and internal datasets for CT-to-MR (sMR) and MR-to-CT (sCT) synthesis.

Main Results:

• The SwinUNETR-based framework consistently outperformed the pix2pix baseline in quantitative metrics (MS-SSIM and PSNR).
• Achieved an MS-SSIM of 0.952 and PSNR of 26.07 dB for sCT synthesis.
• Achieved an MS-SSIM of 0.948 and PSNR of 26.07 dB for sMR synthesis, crucially preserving gray-white matter contrast.

Conclusions:

• Transformer-based architectures like SwinUNETR significantly advance high-fidelity cross-modal medical image synthesis.
• The proposed framework demonstrates superior performance in generating anatomically realistic brain CT and MR images.
• This approach holds great promise for improving the assessment and management of neurodegenerative diseases.