Report on the AAPM grand challenge on deep generative modeling for learning medical image statistics

Area of Science:

• Medical Imaging
• Artificial Intelligence
• Computational Science

Background:

• The 2023 AAPM Grand Challenge focused on deep generative models (DGMs) for learning medical image statistics.
• This report details the findings of the challenge, emphasizing the need for domain-relevant assessments.

Purpose of the Study:

• To advance the development of DGMs for medical imaging applications.
• To establish standardized evaluation metrics for DGM performance in medical image synthesis.
• To highlight the importance of analyzing relevant image statistics for DGM assessment.

Main Methods:

• A common training dataset of ~108,000 512x512 3D breast phantom images was created.
• An evaluation procedure involving a two-stage assessment was developed, including image quality (FID) and reproducibility of image statistics (texture, morphology, etc.).
• Fifty-eight submissions were evaluated, with 9 proceeding to the final ranking based on a summary measure of feature statistics.

Main Results:

• The top-ranked submission utilized a conditional latent diffusion model; runners-up used generative adversarial networks with super-resolution.
• Overall rankings based on comprehensive image statistics differed significantly from FID-based rankings.
• Different DGMs exhibited similar types of artifacts, indicating common challenges in medical image synthesis.

Conclusions:

• Domain-specific evaluation is essential for the effective design and deployment of DGMs in medical imaging.
• The optimal specification for a DGM can vary depending on its intended application and use case.