Generating multi-pathological and multi-modal images and labels for brain MRI.

Area of Science:

• Computer Vision
• Machine Learning
• Artificial Intelligence

Background:

• Generative models are increasingly used to create synthetic data for training deep learning models, offering privacy and data augmentation benefits.
• Current 2D generative methods primarily provide categorical annotations, limiting their use in supervised segmentation tasks.
• Generating paired synthetic images and segmentation maps for 3D applications is an underexplored area.

Purpose of the Study:

• To develop a novel two-stage generative model for synthesizing paired 2D and 3D semantic label maps and multi-modal images.
• To address the gap in generating high-quality, paired synthetic data for supervised segmentation tasks.
• To evaluate the utility of the generated synthetic data in downstream segmentation applications.

Main Methods:

• A two-stage generative approach combining a latent diffusion model for label map synthesis and a Variational Autoencoder Generative Adversarial Network (VAE-GAN) for image synthesis.
• The model generates both semantic label maps and corresponding multi-modal images in 2D and 3D.
• The generated synthetic datasets are evaluated on various segmentation tasks.

Main Results:

• The proposed model successfully generates paired 2D and 3D semantic label maps and multi-modal images.
• Synthetic datasets produced by the model demonstrate effectiveness in diverse segmentation tasks.
• The synthetic data can augment small real datasets or fully replace them while maintaining strong performance.
• The model shows potential to improve performance on out-of-distribution data in downstream tasks.

Conclusions:

• The developed two-stage generative model offers a powerful solution for creating synthetic paired image and segmentation data.
• This approach significantly advances the field of synthetic data generation for deep learning-based segmentation.
• The synthetic data generated shows broad applicability and potential for improving model robustness and performance.