Facilitating Trust Calibration in Artificial Intelligence-Driven Diagnostic Decision Support Systems for Determining...

Area of Science:

• Medical Informatics
• Artificial Intelligence in Medicine
• Clinical Decision Support

Background:

• Diagnostic errors pose significant risks in healthcare.
• Overreliance on artificial intelligence (AI) diagnostic systems can lead to errors.
• Physician trust in AI must be calibrated to actual AI reliability for safe use.

Purpose of the Study:

• To investigate the safe integration of AI diagnostic decision support systems.
• To evaluate trust calibration methods for AI in clinical settings.
• To assess the impact of trust calibration on physician diagnostic accuracy.

Main Methods:

• A quasi-experimental study at Dokkyo Medical University, Japan.
• Physicians were allocated to intervention (trust calibration) and control groups.
• Participants reviewed AI-generated medical histories and differential diagnoses for 20 clinical cases.

Main Results:

• No significant difference in diagnostic accuracy between intervention (41.5%) and control (46%) groups.
• Overall trust calibration accuracy was 61.5%; diagnostic accuracy with correct calibration was 54.5%.
• Trust calibration accuracy significantly predicted physician diagnostic accuracy (aOR 5.90, P<.001).

Conclusions:

• Trust calibration did not significantly enhance diagnostic accuracy in this study.
• Small sample size and suboptimal methods may have limited findings.
• Larger studies and improved trust calibration measures are necessary for safe AI implementation.