Synthesizing breast cancer ultrasound images from healthy samples using latent diffusion models

Breast ultrasound is widely used for cancer screening, but data scarcity and annotation challenges hinder deep learning adoption. Synthetic image generation offers a promising solution to enhance training datasets while preserving patient privacy. However, problems such as inadequate quality of synthesized images and the need for large amounts of data to train the synthesis models remain significant.

We propose a three-stage latent diffusion model (LDM) workflow-enhanced by Vision Transformers and fine-tuned with low-rank adaptation-that synthesizes realistic malignant and benign breast ultrasound images directly from healthy samples while simultaneously generating accurate segmentation masks. Stage division significantly reduces the task complexity of a single synthesis model. Applied to the BUSI dataset (133 healthy, 487 benign, and 210 malignant images), the method generates synthetic cases of each tumor type.

A ResNet101 classifier could not reliably distinguish synthetic from real images (AUC = 0.563), indicating high visual plausibility. Quantitative metrics confirmed strong fidelity: Fréchet inception distance = 15.2 and inception score = 1.79, indicating low distributional divergence in feature space and high similarity to real data. When used for training a U-Net segmentation model, the augmented dataset improved the $F1$ -score from 0.870 to 0.896, demonstrating substantial gains in diagnostic accuracy.

These results show that the proposed three-stage LDM can generate high-quality, anatomically coherent breast cancer images from healthy controls, effectively alleviating data scarcity and enabling more robust training of medical AI systems without compromising clinical realism.