Impact of Responsibility Allocation Structures on Diagnostic Quality in AI-Assisted Diagnosis: Randomized Controlled...

Area of Science:

• Medical Informatics
• Clinical Decision Support Systems
• Human-Computer Interaction

Background:

• Artificial intelligence (AI) is increasingly utilized in clinical diagnosis, yet the division of responsibility between clinicians and AI systems remains undefined.
• Varying responsibility structures can influence clinician judgment and their evaluation of AI-driven diagnostic recommendations.

Purpose of the Study:

• To investigate how different physician-AI responsibility allocation models impact diagnostic accuracy and confidence calibration.
• To evaluate the effectiveness of dynamic, full, and equal responsibility structures compared to a control group in AI-assisted diagnosis.

Main Methods:

• A randomized controlled experiment involving 96 physicians assessing 10 clinical vignettes.
• Four groups were assigned to different responsibility structures: dynamic, full, equal, or control.
• Primary outcomes measured were final diagnostic accuracy and confidence calibration, with secondary outcomes including agreement rates and subjective evaluations.

Main Results:

• The dynamic responsibility structure significantly enhanced diagnostic accuracy (0.717 vs. 0.592) and improved confidence calibration (0.040 vs. 0.150) compared to the control group.
• The full responsibility structure showed no significant difference in accuracy or calibration compared to the control.
• The equal responsibility structure suggested lower diagnostic accuracy and significantly poorer confidence calibration.

Conclusions:

• A dynamic responsibility structure shows promise for optimizing AI integration in healthcare.
• This model may allow healthcare organizations to leverage AI more effectively without negatively impacting clinician diagnostic performance.
• Implementing dynamic responsibility could enhance the safety and quality of AI-assisted diagnostic processes.