Related Experiment Video
Updated: Oct 11, 2025
Brain Infarct Segmentation and Registration on MRI or CT for Lesion-symptom Mapping
Published on: September 25, 2019
Assessing the Trustworthiness of Saliency Maps for Localizing Abnormalities in Medical Imaging
Nishanth Arun1, Nathan Gaw1, Praveer Singh1
1Athinoula A. Martinos Center for Biomedical Imaging, Department of Radiology, Massachusetts General Hospital, Harvard Medical School, 149 13th St, Boston, MA 02129 (N.A., P.S., K.C., M.A., B.C., K.H., S.G., J.P., M.G., M.D.L., J.K.C.); Department of Computer Science, Shiv Nadar University, Greater Noida, India (N.A.); Department of Operational Sciences, Graduate School of Engineering and Management, Air Force Institute of Technology, Wright-Patterson AFB, Dayton, Ohio (N.G.); and Massachusetts Institute of Technology, Cambridge, Mass (K.C., B.C., K.H., J.P., J.A.).
Saliency maps are unreliable for medical image localization. Detection or segmentation models are recommended for accurate abnormality identification, outperforming saliency methods in trustworthiness and performance.
Area of Science:
- Radiology and Artificial Intelligence
- Medical Imaging Analysis
- Computer Vision in Healthcare
Background:
- Saliency maps are used to visualize the regions of medical images that influence a deep learning model's decision.
- Their trustworthiness for abnormality localization in high-stakes medical applications remains under-evaluated.
- Assessing saliency map reliability is crucial for safe and effective clinical implementation.
Purpose of the Study:
- To evaluate the trustworthiness of eight common saliency map techniques for abnormality localization in medical imaging.
- To compare their performance against dedicated localization network architectures.
- To assess sensitivity to model weight randomization, repeatability, and reproducibility.
Main Methods:
- Utilized two large public radiology datasets for pneumothorax segmentation and pneumonia detection.
- Quantified saliency map performance using localization utility (segmentation/detection), sensitivity to model randomization, repeatability, and reproducibility.
- Compared saliency methods against baseline approaches and localization networks (U-Net, RetinaNet) using AUPRC and SSIM metrics.
Main Results:
- All eight saliency map techniques failed at least one trustworthiness criterion and were inferior to localization networks.
- Saliency map AUPRC for segmentation (0.024-0.224) and detection (0.160-0.519) was significantly lower than U-Net (0.404) and RetinaNet (0.596).
- Majority of saliency methods demonstrated poor sensitivity to model randomization, repeatability, and reproducibility compared to localization networks.
Conclusions:
- Saliency maps demonstrate significant limitations in trustworthiness for medical image localization.
- Their use in high-risk medical imaging applications requires careful scrutiny.
- Detection or segmentation models are recommended over saliency maps when localization is the primary network output.
More Related Videos
14:27Identification of Disease-related Spatial Covariance Patterns using Neuroimaging Data
Published on: June 26, 2013
12:50Lesion Explorer: A Video-guided, Standardized Protocol for Accurate and Reliable MRI-derived Volumetrics in Alzheimer's Disease and Normal Elderly
Published on: April 14, 2014