Enhancing AI Clinical Decision Support Trust: Design Workshop Insights from General Practitioners

Area of Science:

• Clinical Informatics
• Artificial Intelligence in Medicine
• Human-Computer Interaction

Background:

• Artificial Intelligence (AI) predictive models are being integrated into Clinical Decision Support Systems (CDSS).
• Healthcare provider trust and acceptance remain significant barriers to the real-world implementation of AI-based CDSS.
• Understanding these barriers is crucial for successful AI adoption in clinical practice.

Purpose of the Study:

• To investigate factors influencing healthcare provider trust and acceptance of AI-based CDSS.
• To explore user, model, and organizational factors affecting clinician trust and recommendation acceptance.
• To identify actionable insights for designing AI systems that foster trust.

Main Methods:

• A workshop was conducted with General Practitioners (GPs).
• Discussions focused on user, model, and organizational factors impacting trust and acceptance of AI-CDSS.
• Qualitative insights were gathered from clinician participants.

Main Results:

• Explainability is vital, but current Explainable AI (XAI) visualizations offer limited value to non-technical clinicians.
• Clinicians desire transparency in AI model training data (sources, elements).
• Alignment of AI recommendations with existing clinical research and guidelines is essential for trust.

Conclusions:

• Enhanced AI system design requires more than just technical explainability.
• Transparency in data and alignment with clinical evidence are critical for clinician trust and acceptance of AI-CDSS.
• Future AI-CDSS development should prioritize user-centered design focusing on trust-building elements.