How Explainable Artificial Intelligence Can Increase or Decrease Clinicians' Trust in AI Applications in Health Care:...

Area of Science:

• Medical Informatics
• Artificial Intelligence in Healthcare
• Human-Computer Interaction

Background:

• Artificial intelligence (AI) offers significant potential in clinical practice, but its
• black box
• nature can hinder clinician adoption.
• Explainable AI (XAI) aims to build trust by providing insights into AI decision-making processes.
• Ensuring appropriate clinician trust is crucial for effective AI integration, avoiding both over-reliance and skepticism.

Purpose of the Study:

• To systematically review and synthesize empirical evidence on how Explainable AI (XAI) impacts clinician trust in AI-driven clinical decision-making.
• To understand the nuances of trust modulation through AI explanations in healthcare settings.
• To identify factors influencing the effectiveness of XAI in fostering appropriate levels of clinician trust.

Main Methods:

• Systematic review conducted following PRISMA guidelines, searching PubMed and Web of Science.
• Inclusion criteria focused on empirical studies measuring XAI's impact on clinician trust (cognitive or affect-based measures).
• 10 studies were included out of 778 screened articles, with risk of bias assessed for each.

Main Results:

• The majority of studies (5/10) found that XAI increased clinician trust compared to standard AI, particularly with clear, concise, and clinically relevant explanations.
• Three studies reported no significant effect of XAI on trust, indicating explanations do not automatically improve it.
• Two studies showed XAI could either enhance or diminish trust based on explanation complexity and coherence, emphasizing the critical role of explanation quality.

Conclusions:

• The quality and clarity of explanations significantly modulate clinician trust in AI, with complex or contradictory explanations potentially undermining it.
• Achieving an appropriate balance of trust is essential, preventing both blind faith and undue skepticism towards AI recommendations.
• Further research is needed to refine trust measures and develop strategies for optimal AI integration in clinical practice.