Identifying Deprescribing Opportunities With Large Language Models in Older Adults: Retrospective Cohort Study.

Area of Science:

• Artificial Intelligence in Medicine
• Geriatric Pharmacology
• Emergency Medicine

Background:

• Polypharmacy in older adults increases adverse drug events, including falls.
• Deprescribing aims to mitigate these risks by discontinuing inappropriate medications.
• Current deprescribing criteria are challenging to apply in time-constrained emergency settings.

Purpose of the Study:

• Evaluate a large language model (LLM) pipeline for identifying deprescribing opportunities in older emergency department (ED) patients.
• Assess LLM performance using Beers, STOPP, and GEM recommendations criteria.
• Examine LLM confidence calibration and its impact on recommendation accuracy.

Main Methods:

• Retrospective cohort study of 100 older adults in a US academic medical center ED.
• LLM pipeline filtered deprescribing criteria and applied them using patient data.
• Model recommendations compared to medical students, with discrepancies adjudicated by ED physicians.

Main Results:

• LLM effectively identified deprescribing criteria (PPV 0.83, NPV 0.93) compared to medical students.
• LLM struggled with specific deprescribing recommendations (PPV 0.47, NPV 0.93) due to complex criteria and missing data.
• Selective prediction showed limited improvement due to poorly calibrated LLM confidence.

Conclusions:

• LLMs can aid in filtering deprescribing criteria for ED use.
• Challenges persist in applying criteria to complex clinical scenarios and ensuring accurate LLM confidence.
• Improved guidelines, LLM calibration, and human-AI integration are crucial for effective clinical implementation.