Large Language Model Influence on Diagnostic Reasoning: A Randomized Clinical Trial.

Area of Science:

• Medical Artificial Intelligence
• Clinical Decision Support Systems
• Physician Performance Evaluation

Background:

• Large language models (LLMs) show promise in medical reasoning assessments.
• The impact of LLMs on actual physician diagnostic reasoning remains unclear.

Purpose of the Study:

• To evaluate the effect of large language models (LLMs) on physician diagnostic reasoning compared to traditional resources.

Main Methods:

• A single-blind randomized clinical trial involved 50 physicians across multiple institutions.
• Participants were randomized to use LLMs with conventional resources or conventional resources alone.
• Diagnostic performance was assessed using a standardized rubric and expert consensus.

Main Results:

• No significant difference in diagnostic reasoning scores was found between the LLM group and the conventional resources group (76% vs. 74%).
• Time spent per case did not differ significantly between groups.
• The LLM, when used alone, scored significantly higher (16 percentage points) than the conventional resources group.

Conclusions:

• Integrating LLMs as diagnostic aids did not enhance physician clinical reasoning in this study.
• LLMs alone demonstrated superior performance, highlighting the need for further development in AI-physician collaboration.
• Future research should focus on optimizing the synergy between artificial intelligence and clinical practice.