Assessing the Limitations of Large Language Models in Clinical Practice Guideline-Concordant Treatment...

Area of Science:

• Artificial Intelligence in Medicine
• Clinical Decision Support Systems

Background:

• Large language models (LLMs) show promise in therapeutic decision-making, but prior studies used highly curated patient data.
• The effectiveness of LLMs in real-world clinical scenarios with unstructured data remains largely uninvestigated.

Purpose of the Study:

• To evaluate the ability of various large language models (LLMs) to make guideline-concordant treatment decisions for severe aortic stenosis using typical clinical data.
• To assess LLM performance based on the input data format (unstructured medical reports vs. case summaries) and prompt augmentation.

Main Methods:

• A retrospective study of 80 severe aortic stenosis patients treated in 2022.
• Multiple LLMs were queried using anonymized original medical reports and manually generated case summaries.
• Performance was measured by agreement with institutional heart team decisions (Cohen κ), reliability (ICC), and fairness (FBI).

Main Results:

• LLMs demonstrated poor performance with original, unstructured medical reports (Cohen κ: -0.47 to 0.22).
• Performance significantly improved with case summaries and added guideline knowledge (Cohen κ: -0.02 to 0.63).
• All tested LLMs exhibited hallucinations, and bias towards TAVR was observed (FBI: 0.46-1.51).

Conclusions:

• Advanced LLMs require meticulously curated input for reliable clinical decision-making.
• The presence of hallucinations and bias necessitates caution when implementing LLMs in real-world healthcare settings.
• Further research is needed to enhance LLM accuracy and safety for clinical applications.