Leveraging Clinical Text and Class Conditioning for 3D Prostate MRI Generation

Area of Science:

• Medical Imaging
• Machine Learning
• Artificial Intelligence

Background:

• Latent diffusion models (LDMs) show promise for medical imaging but face challenges with data scarcity.
• Current medical LDM strategies often require large datasets, specific text encoders, or non-medical models, limiting performance and accessibility.
• There is a need for data-efficient and accessible LDM methods in medical imaging.

Purpose of the Study:

• To propose a novel LDM conditioning approach to address data scarcity and accessibility limitations in medical imaging.
• To introduce a data-efficient pipeline for training LDMs using limited data and minimal annotation.
• To improve both synthetic medical image quality and the performance of downstream machine learning tasks.

Main Methods:

• Developed Class-Conditioned Efficient Large Language model Adapter (CCELLA), a dual-head conditioning approach for LDMs.
• CCELLA simultaneously conditions the LDM U-Net with free-text clinical reports and radiology classifications.
• Implemented a data-efficient LDM pipeline centered around CCELLA with a joint loss function, evaluated on 3D prostate MRI.

Main Results:

• Achieved a 3D FID score of 0.025 on a limited 3D prostate MRI dataset, outperforming a foundation model (FID 0.070).
• Augmenting classifier training with synthetic images improved prostate cancer prediction accuracy from 69% to 74%.
• Classifier performance using only synthetic images approached that of real image training.

Conclusions:

• The proposed CCELLA-centric pipeline enables high-quality medical image synthesis with limited data and annotation.
• This method enhances both synthetic image quality and downstream classifier performance.
• The approach improves LDM performance and scientific accessibility in medical imaging.