Human-machine teaming is key to AI adoption: clinicians' experiences with a deployed machine learning system

Area of Science:

• Clinical Informatics
• Artificial Intelligence in Healthcare
• Human-Computer Interaction

Background:

• Machine learning (ML) systems are increasingly implemented in clinical settings to enhance patient care.
• Widespread adoption and realization of ML's potential in healthcare remain challenging.
• Understanding clinician interaction with ML tools is crucial for successful integration.

Purpose of the Study:

• To explore clinicians' perceptions and experiences with an ML-based system for sepsis detection.
• To identify factors influencing clinician trust and adoption of ML tools in clinical practice.

Main Methods:

• Qualitative analysis of coded interviews with clinicians using an ML system for sepsis.
• Focus on understanding the clinician-technology relationship and trust-building mechanisms.

Main Results:

• Clinicians perceive ML systems as partners, not replacements for their clinical judgment.
• Trust in ML systems is built through hands-on experience, expert validation, and system design.
• Systems that support clinician autonomy and integrate seamlessly into workflows foster greater trust and adoption.

Conclusions:

• Successful ML adoption in healthcare hinges on fostering a collaborative relationship between clinicians and technology.
• Designing ML systems that respect clinical workflows and autonomy is key to building trust.
• Clinician trust in ML can be cultivated even without deep technical understanding, emphasizing practical validation and supportive system design.