Effects of explainable artificial intelligence in neurology decision support

Area of Science:

• Medical Informatics
• Artificial Intelligence in Healthcare
• Human-Computer Interaction

Background:

• Artificial intelligence (AI)-based decision support systems (DSS) are increasingly used in medicine, but their internal decision-making processes often remain opaque.
• Explainable AI (xAI) techniques aim to provide transparency into these AI systems, yet optimal design for clinical users is not well understood.

Purpose of the Study:

• To investigate the impact of various xAI techniques on clinician interaction with AI-based DSS.
• To compare the effects of xAI on medical professionals versus the general population in decision-making tasks.

Main Methods:

• A randomized, blinded study compared neurologists to a general population using an AI-DSS with different xAI interventions (e.g., decision trees, feature importance, no explanation).
• Primary outcomes assessed included test performance and user perceptions of explainability, trust, and social competence.
• Secondary outcomes measured compliance, understandability, and agreement with DSS recommendations.

Main Results:

• Decision trees were perceived as more explainable by neurologists than the general population and probability scores.
• Increased neurology experience and perceived explainability were associated with degraded performance.
• Individual xAI methods did not predict performance; perceived explainability was the key factor.

Conclusions:

• xAI methods have differential effects on medical professionals compared to the general population, indicating no one-size-fits-all solution.
• Further user-centered research is required to develop personalized AI decision support systems tailored for clinicians.