Multi-modal MRI synthesis with conditional latent diffusion models for data augmentation in tumor segmentation

Area of Science:

• Medical Imaging
• Deep Learning
• Computer Vision

Background:

• Multimodality is crucial for accurate medical image segmentation, particularly for multilabel tasks like tumor segmentation.
• Limited annotated medical data and the complexity of volumetric data pose challenges for deep learning models.
• Conventional data augmentation techniques are often inadequate for 3D medical imaging.

Purpose of the Study:

• To propose a novel slice-based latent diffusion architecture for generating 3D multi-modal images and multi-label masks.
• To address the challenge of limited annotated training data in medical imaging.
• To improve the efficiency and performance of tumor segmentation tasks.

Main Methods:

• Developed a slice-based latent diffusion architecture for simultaneous image and mask generation.
• Incorporated positional encoding and a Latent Aggregation module for spatial coherence and slice sequentiality.
• Utilized conditional generation based on tumor characteristics and a refining module for texture enhancement.

Main Results:

• The proposed method effectively reduces computational complexity and memory demands.
• Synthesized volumes demonstrated superior performance and efficiency in downstream tumor segmentation tasks compared to state-of-the-art diffusion models.
• The approach mitigates blurriness in generated images caused by data scarcity.

Conclusions:

• The slice-based latent diffusion architecture offers an efficient and effective solution for generating multi-modal medical images and masks.
• This method significantly enhances tumor segmentation accuracy, with potential applications in clinical diagnosis and treatment planning.
• The architecture is adaptable to other medical imaging modalities beyond tumor segmentation.