Explainability does not improve biochemistry staff trust in artificial intelligence-based decision support.

Area of Science:

• Healthcare AI
• Clinical Decision Support Systems
• Human-Computer Interaction

Background:

• Explainability in artificial intelligence-based decision support (ADS) systems aims to clarify prediction reasoning.
• A common assumption is that explainability enhances user trust, particularly in healthcare, but this remains unproven.
• Generating explanations for complex algorithms like artificial neural networks adds significant complexity.

Purpose of the Study:

• To investigate whether explainability in ADS increases user trust and improves performance in a healthcare setting.
• To evaluate if the benefits of explainability justify the added computational complexity.

Main Methods:

• Biochemistry staff performed a wrong blood in tube (WBIT) error identification task using an ADS.
• Participants were divided into two groups: one received ADS predictions with explanations, the other received predictions alone.
• User trust was indexed by agreement with ADS predictions, and performance was measured by WBIT error detection accuracy.

Main Results:

• No significant difference in agreement rates with ADS predictions was found between groups (83.3% vs. 81.8%, p=0.46).
• Both groups showed equivalent performance in WBIT error identification, including accuracy, sensitivity, and specificity (p-values >0.78).

Conclusions:

• The study found no evidence that explainability increased user trust or improved performance in the WBIT error identification task.
• The additional complexity of generating explanations is not justified by enhanced user trust in this specific healthcare application.