Identify Patients at Risk of HIV Using a Clinical Large Language Model from Electronic Health Records.

Area of Science:

• Computational biology
• Medical informatics
• Public health

Background:

• Electronic health records (EHRs) contain valuable data for identifying individuals at risk for diseases like HIV.
• Traditional machine learning models have been used for risk prediction, but LLMs offer new possibilities for analyzing complex clinical data.

Purpose of the Study:

• To develop and evaluate a large language model (LLM)-based solution for identifying individuals at HIV risk using structured EHR data.
• To compare the performance of the LLM approach with traditional machine learning models.

Main Methods:

• Structured EHR data (demographics, diagnoses, medications) were transformed into narrative descriptions.
• GatorTron, a clinical LLM, was applied to the narrative data.
• Performance was compared against traditional models like LASSO and XGBoost using metrics such as F1 score and AUC.

Main Results:

• The LLM solution achieved an F1 score of 53.5% and an AUC of 0.88, comparable to traditional machine learning models.
• Both LLM and traditional models demonstrated AUCs above 0.82 across various demographic subgroups (age, sex, race/ethnicity).
• Interpretability analyses revealed consistent patterns between LLM and traditional models.

Conclusions:

• LLM-based solutions, like the one using GatorTron, are effective for identifying individuals at HIV risk from EHR data.
• The performance of LLM approaches is comparable to established machine learning methods.
• These findings support the use of LLMs in clinical risk prediction and public health initiatives.