Clinical Sentiment Analysis by Large Language Models Enhances Prediction of Hepatorenal Syndrome

Hepatorenal syndrome-acute kidney injury (HRS-AKI) is a severe complication of decompensated cirrhosis that is challenging to predict. Large language models are robust for natural language processing (NLP). We aimed to determine if clinical sentiment in unstructured text can be derived by large language models and used to improve prediction of HRS-AKI over traditional predictors alone.

Cohort study consisting of adult patients admitted to a single academic medical center with decompensated cirrhosis and AKI. We used Microsoft Azure OpenAI generative pretrained transformer (GPT)-4o to derive a sentiment score ranging from 0 to 1 for HRS-AKI and conduct NLP extraction of clinical terms associated with HRS-AKI in admission notes. The area under the receiver operator curve (AUROC) was compared in logistic regression models incorporating structured variables with versus without sentiment scores and NLP-extracted clinical terms.

In our cohort of 314 participants, the median age was 60, 48% were female, and the median Model for End Stage Liver Disease (MELD) was 31. Higher sentiment score was associated with the diagnosis of HRS-AKI (odds ratio: 1.52 per 0.1; 95% confidence interval: 1.12-2.14) in multivariate models. AUROC using structured clinical covariates alone was 0.63. With the addition of GPT-4o-derived sentiment score and clinical terms to structured covariates, the final model yielded an improved AUROC of 0.79 (P < .01).

Clinical texts contain data that are difficult to extract using standard methodologies. Sentiment analysis and NLP-based variable derivation with GPT-4o in clinical application is feasible and can improve the prediction of HRS-AKI over structured data alone.