Related Experiment Video
Updated: Jun 29, 2025
03:14
Augmenting Large Language Models via Vector Embeddings to Improve Domain-Specific Responsiveness
Published on: December 6, 2024
554
Using generative AI to investigate medical imagery models and datasets
Oran Lang1, Doron Yaya-Stupp1, Ilana Traynis2
1Google, Mountain View, CA, USA.
Ebiomedicine
|April 2, 2024
Summary
This study introduces a new AI method to explain medical image classifications by visualizing learned visual attributes. The approach helps uncover clinically relevant signals and potential confounders, aiding trust and discovery in AI-driven healthcare.
Area of Science:
- Medical Imaging AI
- Explainable AI (XAI)
- Computer Vision in Healthcare
Background:
- AI models show promise in medical imaging but lack explainability.
- Understanding AI decision-making is crucial for clinical trust and novel discoveries.
- Current AI explanations are insufficient for complex medical tasks.
Purpose of the Study:
- To develop a workflow for generating hypotheses about visual signals AI models learn for medical image classification.
- To enhance trust in AI by explaining its predictions and uncovering new insights.
- To enable scientific discovery by identifying unknown signals in medical data.
Main Methods:
- A 4-step workflow involving classifier training, StyleGAN-based image generation (StylEx), automatic attribute visualization, and interdisciplinary expert review.
- Generating counterfactual visualizations by modifying detected attributes to understand their impact on predictions.
- Presenting discovered attributes and visualizations to experts to formulate hypotheses on underlying mechanisms.
Main Results:
- Demonstrated applicability across eight tasks and three medical imaging modalities (retinal, eye, chest).
- Identified clinically known features, automatically-learned confounders (e.g., X-ray underexposure, eye makeup), and novel, physiologically plausible attributes (e.g., sex-related fundus differences).
Conclusions:
- The approach facilitates hypothesis generation for better understanding, assessment, and knowledge extraction from AI models.
- Highlights the importance of interdisciplinary perspectives to interpret attributes reflecting real-world healthcare delivery and socio-cultural factors.
- Code will be released to enable researchers to train StylEx models and responsibly interpret revealed attributes.
