Lung cancer CT image generation from a free-form sketch using style-based pix2pix for data augmentation

Area of Science:

• Medical Imaging
• Artificial Intelligence
• Computer Vision

Background:

• Artificial intelligence (AI) in medical imaging requires large datasets, which are difficult to obtain.
• Generative adversarial networks (GANs) offer a potential solution through data augmentation with synthetic images.
• Existing GANs face challenges in generating high-quality and diverse images for effective augmentation.

Purpose of the Study:

• To address the limitations of current GANs in medical image data augmentation.
• To develop a novel GAN-based model for generating diverse, free-form lesion images from tumor sketches.
• To enable one-to-many image generation for enhanced data augmentation in medical AI.

Main Methods:

• Proposed StylePix2pix, an improved pix2pix model enabling one-to-many image generation.
• Incorporated a mapping network and style blocks inspired by StyleGAN.
• Utilized 20 tumor sketches created by a physician for image generation.

Main Results:

• The StylePix2pix model successfully generated free-form lesion images from tumor sketches.
• The generated images accurately reproduced complex tumor shapes.
• Demonstrated the capability for one-to-many image generation, crucial for data augmentation.

Conclusions:

• StylePix2pix offers a promising approach for generating diverse medical images.
• The model's one-to-many generation capability enhances its suitability for AI data augmentation.
• This method can help overcome data scarcity challenges in medical AI applications.