Which explainable AI methods in medical imaging are clinically impactful? A systematic literature review addressing...

Explainable Artificial Intelligence (XAI) has emerged as a strategy to enhance the transparency and interpretability of AI systems in medical imaging. Although numerous methods have been developed to generate explanations of model behavior, their evaluation has predominantly relied on technical performance metrics rather than clinician-centered assessment. The limited involvement of clinicians in the development and validation of XAI methods, together with the absence of clinically meaningful evaluation frameworks, represents a significant barrier to the successful integration of AI into routine clinical workflows.

To conduct a comprehensive review of the existing literature on the clinician-centered evaluation of XAI techniques in the domain of medical imaging.

This systematic review was conducted in accordance with the Preferred Reporting Items for Systematic Reviews and Meta-Analyzes (PRISMA 2020) guidelines. A literature search (in Medline, Web of Science, IEEE, ACM Digital Library and Scopus) was performed from inception up to November 17, 2025 in collaboration with a medical information specialist. The study protocol was prospectively registered in the International Prospective Register of Systematic Reviews (PROSPERO; registration number CRD420261301196). Any modifications to the original protocol are recorded in the PROSPERO entry and are reported in this manuscript where applicable. Study designs were categorized using MMAT and risk of bias was assessed with a sample-size adjustment.

We identified 9,305 records from five databases, which were reduced to 5,687 after removing duplicates. Following title and abstract screening, 5,440 articles were excluded as irrelevant. Full-text assessment led to the exclusion of 192 articles, primarily because they did not involve healthcare professionals in evaluating explainability (90 studies) or were unrelated to medical imaging (73 studies). Ultimately, 51 studies met the inclusion criteria and were independently reviewed, and bibliographic details and key contributions to XAI in medical imaging were extracted.

Clinician-centered evaluation of XAI in medical imaging is expanding but remains methodologically fragile. The available evidence suggests that the type of explanation may influence evaluation outcomes. In several studies, example-based and concept-based methods are associated with improvements in both subjective and objective measures, as well as with assessments of automation bias. In contrast, attribution-based explanations are more frequently linked to enhanced clinician perceptions, while their relationship with decision-making outcomes and automation bias remains less clear.

https://www.crd.york.ac.uk/PROSPERO/view/CRD420261301196.