Related Experiment Video
Updated: Jul 6, 2025
Author Spotlight: Enhancement of Salient Object Detection for Smart Grid Applications
Published on: December 15, 2023
Revisiting the Trustworthiness of Saliency Methods in Radiology AI
Jiajin Zhang1, Hanqing Chao1, Giridhar Dasegowda1
1From the Department of Biomedical Engineering, Center for Biotechnology and Interdisciplinary Studies, Rensselaer Polytechnic Institute, 110 8th St, Biotech 4231, Troy, NY 12180 (J.Z., H.C., G.W., P.Y.); and Department of Radiology, Massachusetts General Hospital, Harvard Medical School, Boston, Mass (G.D., M.K.K.).
Saliency maps in radiology artificial intelligence (AI) show low sensitivity and robustness to input changes, potentially misleading interpretations. The prediction-saliency correlation (PSC) metric helps validate AI explainability trustworthiness.
Area of Science:
- Medical Imaging
- Artificial Intelligence in Radiology
- Explainable AI
Background:
- Saliency maps in AI are crucial for understanding model decisions in medical imaging.
- Assessing the robustness of these maps to input perturbations is vital for clinical trust.
Purpose of the Study:
- To evaluate the sensitivity and robustness of saliency maps in AI radiology to subtle input perturbations.
- To determine if these perturbations can lead to misleading interpretations of AI outputs.
Main Methods:
- Utilized prediction-saliency correlation (PSC) to assess seven common saliency methods on chest radiographs and brain tumor MR images.
- Employed a model-agnostic approach and conducted a reader study with radiologists on perturbed images.
Main Results:
- Saliency methods exhibited low sensitivity (max PSC 0.25) and weak robustness (max PSC 0.12) on the CheXpert dataset.
- Saliency maps from a commercial prototype were found to be irrelevant to model output, with expert identification accuracy below 44.8% for perturbed images.
Conclusions:
- Popular saliency methods demonstrate poor sensitivity and robustness, questioning their reliability in medical AI.
- The proposed PSC metric offers a quantitative tool for validating the trustworthiness of AI explainability in radiology.
