Deep learning based synthesis of MRI, CT and PET: Review and analysis

Area of Science:

• Medical imaging and artificial intelligence.
• Computer vision and machine learning applications in healthcare.

Background:

• Medical image synthesis aims to generate one image modality from another (e.g., CT, MRI, PET) to enhance clinical decision-making and overcome acquisition challenges.
• Deep learning generative models show superior performance in medical image translation compared to traditional methods due to complex, non-linear domain mappings.

Purpose of the Study:

• To comprehensively review deep learning-based medical imaging translation techniques from 2018 to 2023.
• To analyze various synthesis methods, network architectures (CNNs, Transformers, Diffusion models), loss functions, datasets, and performance metrics.
• To identify current challenges and suggest future research directions in the field of synthetic medical image generation.

Main Methods:

• Systematic literature review of deep learning-based medical image synthesis methods.
• Analysis of network architectures, including Convolutional Neural Networks (CNNs), Transformers, and Diffusion models.
• Comparative assessment of loss functions, datasets, anatomical regions, image quality, and downstream task performance.

Main Results:

• Deep learning models, particularly generative ones, have significantly advanced medical image synthesis for pseudo-CT, synthetic MR, and synthetic PET.
• Novel architectures like Transformers and Diffusion models show promise beyond conventional CNNs.
• Performance varies based on model design, loss functions, datasets, and anatomical focus, with image quality and downstream task utility being key evaluation criteria.

Conclusions:

• Deep learning offers powerful tools for medical image synthesis, addressing modality acquisition limitations.
• Continued research into novel architectures and standardized evaluation is crucial for clinical translation.
• This survey provides a roadmap for researchers navigating the evolving landscape of medical image synthesis.