Inflating 2D convolution weights for efficient generation of 3D medical images

Area of Science:

• Medical Imaging
• Artificial Intelligence
• Computer Vision

Background:

• Three-dimensional (3D) medical image generation holds significant potential for understanding anatomical structures.
• Effective training of 3D generative models is hindered by the high cost of data acquisition/annotation and the computational complexity of 3D convolutions.

Purpose of the Study:

• To propose a novel, parameter-efficient Generative Adversarial Network (GAN) for 3D medical image synthesis.
• To address challenges of data scarcity and high parameter counts in 3D medical image generation.

Main Methods:

• Introduced the 3D Split&Shuffle-GAN model.
• Pre-trained a 2D GAN on abundant image slices and inflated weights for 3D GAN initialization to mitigate data scarcity.
• Developed novel, parameter-efficient 3D network architectures for the generator and discriminator.

Main Results:

• Achieved improved 3D image generation quality, demonstrated by a 14.7 improvement in Fréchet Inception Distance on heart and brain datasets.
• Significantly reduced model parameters, utilizing only 48.5% of those in the baseline method.
• Validated effectiveness on the Stanford AIMI Coronary Calcium and Alzheimer's Disease Neuroimaging Initiative datasets.

Conclusions:

• Successfully developed a parameter-efficient 3D medical image generation model.
• The proposed method demonstrates efficiency and effectiveness in generating high-quality 3D medical images.
• Potential applications include generating 3D brain and heart images for real-world clinical use cases.