LLM-Powered Automated Infection Detection in Cirrhosis: Achieving Expert-Level Accuracy from Clinical Documents

Systemic infections, particularly bacterial etiologies, are the leading cause of hospitalization, decompensation, and mortality in patients with cirrhosis, emphasizing the need for accurate and timely identification. Reliable infection labeling is crucial for developing clinical decision support tools and predictive models. However, existing methods, such as ICD codes and manual chart reviews, are unreliable, labor-intensive, and lack scalability. This study leverages a state-of-the-art large language model (LLM), Claude 3.5 Sonnet, deployed on a HIPAA-compliant AWS instance, to automate infection identification from the first 48 hours of clinical notes in hospitalized patients with cirrhosis. Gold-standard infection labels were created through manual physician chart reviews. Results from 196 cases demonstrated the LLM's robust performance, achieving 89.8% accuracy, 97.9% sensitivity, and a PPV of 89.1% compared to physician chart review. Additionally, the model identified multiple pathogen types and infection locations in complex cases. These findings validate the potential of LLMs as scalable and efficient tools for infection classification, paving the way for AI-driven methodologies in infection management.Clinical Relevance- This study demonstrates the potential of large language models to automate infection identification and classification in patients with cirrhosis, improving accuracy and scalability beyond traditional methods.