Clinician perspectives on explainability in AI-driven closed-loop neurotechnology.

Area of Science:

• Neurotechnology
• Artificial Intelligence
• Clinical Translation

Background:

• Artificial Intelligence (AI) shows potential for neurotechnology advancement and clinical application.
• AI-driven neurotechnologies use complex algorithms for brain data analysis and closed-loop neurostimulation.
• Limited clinical integration of AI is often due to a lack of explainability.

Purpose of the Study:

• To investigate clinician attitudes towards AI-driven closed-loop neurotechnologies.
• To explore clinicians' informational needs and preferences regarding AI explainability.
• To determine necessary forms of explanation for clinical AI adoption.

Main Methods:

• Conducted semi-structured expert interviews with 20 clinicians (neurologists, neurosurgeons, psychiatrists) in Germany and Switzerland.
• Employed reflexive thematic analysis to understand clinician expectations for AI explainability.
• Focused on AI in closed-loop neurotechnology systems.

Main Results:

• Clinicians prioritize context-sensitive, clinically meaningful explanations (e.g., input data, outcome relevance).
• Detailed technical model information was of low interest to clinicians.
• Clinicians specifically requested Explainable AI (XAI) techniques like feature importance.

Conclusions:

• Clinical utility of AI neurotechnologies can be enhanced by focusing on intuitive, user-centered, and clinically relevant explainability.
• Prioritizing pragmatic clinician needs over full algorithmic transparency can bridge the AI development-to-clinical implementation gap.
• Designing AI systems with clinician-focused explainability is crucial for successful adoption.