TCGAN: a transformer-enhanced GAN for PET synthetic CT.

Area of Science:

• Medical imaging
• Artificial Intelligence
• Computer Vision

Background:

• Multimodal medical images aid diagnostics but are costly and pose safety concerns.
• Medical image synthesis offers solutions, with generative adversarial networks (GANs) showing promise.
• Existing GAN-based methods for missing modality synthesis have limitations.

Purpose of the Study:

• To propose a novel generator combining transformer and convolutional neural network (CNN) architectures for enhanced medical image synthesis.
• To address limitations in current generative adversarial network (GAN)-based approaches for synthesizing missing image modalities.
• To improve the detail and quality of synthesized medical images.

Main Methods:

• Developed a hybrid generator integrating transformer networks and CNNs.
• Applied the method to positron emission tomography (PET) to computed tomography (CT) synthesis for attenuation correction.
• Validated the approach on magnetic resonance (MR) T1- to T2-weighted image synthesis tasks.

Main Results:

• The proposed hybrid network effectively synthesizes missing medical image modalities.
• Demonstrated superior performance in PET to CT synthesis for attenuation correction.
• Achieved state-of-the-art results in MR T1- to T2-weighted image synthesis based on qualitative and quantitative analyses.

Conclusions:

• The combined transformer and CNN generator offers a powerful approach for medical image synthesis.
• The method shows significant potential for applications like PET attenuation correction and cross-modality MR synthesis.
• This hybrid architecture advances the field of medical image synthesis, improving diagnostic capabilities.