Development and validation of an AI algorithm to generate realistic and meaningful counterfactuals for retinal...

Area of Science:

• Medical Imaging AI
• Ophthalmology AI
• Computer Vision in Medicine

Background:

• Counterfactual reasoning is vital in clinical settings, especially for image-based specialties like ophthalmology.
• AI-generated counterfactual medical images can enhance clinician training and patient education by visualizing hypothetical scenarios.

Purpose of the Study:

• To develop an AI model capable of generating realistic counterfactual retinal images (fundus photography and OCT).
• To illustrate potential disease states or changes in retinal images in response to "what if" questions.

Main Methods:

• Trained classifiers on large-scale retinal image datasets (CFP and OCT) for disease classification.
• Developed an unconditional diffusion model to generate diverse retinal images, including those with lesions.
• Combined diffusion models with classifier guidance to produce realistic counterfactual images preserving retinal structure.

Main Results:

• The AI successfully generated counterfactuals by adding or removing disease-specific features.
• Generated color fundus photography (CFP) images were indistinguishable from real images and clinically relevant.
• Generated optical coherence tomography (OCT) images were realistic but identifiable by experts with higher than chance accuracy.

Conclusions:

• Combining diffusion models with classifier guidance effectively creates realistic and meaningful counterfactuals for high-resolution medical images like CFP.
• The generated counterfactual images hold potential for medical professional training and patient education in ophthalmology.