Evaluating Explainability: A Framework for Systematic Assessment of Explainable AI Features in Medical Imaging

Area of Science:

• Medical Imaging
• Artificial Intelligence
• Explainable AI

Background:

• Explainability features in AI devices offer insights into internal mechanisms.
• Current evaluation techniques for AI explanations are lacking, especially in medical imaging.
• A need exists for robust methods to assess the quality of AI-generated explanations in healthcare.

Purpose of the Study:

• To propose a comprehensive framework for assessing and reporting explainable AI (XAI) features in medical images.
• To establish quantifiable criteria for evaluating the quality of explanations provided by AI in medical devices.
• To develop a scorecard for XAI methods in medical imaging to accompany AI devices.

Main Methods:

• Developed an evaluation framework based on four criteria: consistency, plausibility, fidelity, and usefulness.
• Defined consistency as the variability of explanations for similar inputs.
• Defined plausibility as the closeness of the explanation to ground truth, fidelity as the alignment with model mechanisms, and usefulness as the impact on task performance.

Main Results:

• The framework provides a quantitative method to assess AI explanation quality in medical imaging.
• A scorecard was developed for a complete description and evaluation of XAI methods.
• Case study using Ablation CAM and Eigen CAM illustrated heatmap evaluation for breast lesion detection.

Conclusions:

• The proposed framework establishes criteria for quantifying the quality of explanations from medical AI devices.
• This work addresses the lack of evaluation techniques for XAI in medical imaging.
• The developed scorecard and criteria facilitate a thorough assessment of AI explainability in clinical applications.