Identification of Long-Term Care Facility Residence From Admission Notes Using Large Language Models

Area of Science:

• Health Informatics
• Artificial Intelligence in Medicine
• Infectious Disease Prevention

Background:

• Antimicrobial-resistant organisms colonize approximately 50% of long-term care facility (LTCF) residents.
• Early identification of LTCF-exposed patients in acute care hospitals is crucial for preventing intrahospital spread.
• Current electronic health records often fail to capture LTCF exposure, leading to undetected high-risk patients.

Purpose of the Study:

• To evaluate the performance of a large language model (LLM) in identifying recent LTCF exposure from patient admission histories.
• To compare the LLM's accuracy and efficiency against human review.
• To assess the cost-effectiveness of using LLMs for this task.

Main Methods:

• A cross-sectional, multicenter study analyzed history and physical (H&P) notes from 2087 adult admissions across 13 hospitals.
• A large language model (GPT-4-Turbo) was employed using zero-shot learning and prompting to identify recent LTCF exposure (≤12 months).
• LLM performance (sensitivity, specificity) was compared against human adjudication, with secondary analysis of review time and cost.

Main Results:

• The LLM demonstrated high accuracy, achieving 97% sensitivity and 98% specificity at one institution and 96% sensitivity and 93% specificity at another.
• LLM review was significantly faster (4-6 seconds per note) and less expensive ($0.03 per note) compared to human review (2.5 minutes per note, $0.63-$0.83).
• LLM-generated rationales were factually correct, accurately quoted note text, and occasionally demonstrated inferential logic; 37 human errors were identified.

Conclusions:

• Large language models (LLMs) can accurately identify long-term care facility (LTCF) exposure from clinical notes.
• LLM-based review is substantially more efficient and cost-effective than manual human review.
• Implementing LLMs offers a promising strategy to improve the detection of high-risk patients for infection control.