Evaluating Explanations From AI Algorithms for Clinical Decision-Making: A Social Science-Based Approach

Area of Science:

• Artificial Intelligence
• Clinical Decision Support Systems
• Explainable AI (XAI)

Background:

• Explainable Artificial Intelligence (XAI) provides reasons for AI model predictions.
• Evaluating AI explanations involves assessing faithfulness and usefulness.
• Automated metrics for explanation usefulness in clinical settings are lacking.

Purpose of the Study:

• To develop a novel metric for evaluating the usefulness of AI explanations for clinicians.
• To create a scoring method considering human cognition and clinical requirements for AI explanations.

Main Methods:

• Developed a scoring method for XAI explanations providing feature importance values.
• Integrated social science theories and biomedical knowledge graphs for usefulness assessment.
• Evaluated the method using a case study on sepsis onset prediction in ICUs.

Main Results:

• The new metric's scores align with clinical literature evidence.
• The developed metric demonstrates qualities suitable for evaluating AI explanation usefulness.
• The method successfully quantifies the support for explanations from clinical literature.

Conclusions:

• The proposed metric can evaluate and select useful AI explanations in clinical contexts.
• This tool is fundamental for designing effective AI-driven clinical decision support systems.
• Facilitates the advancement of trustworthy and clinically relevant AI in healthcare.