A Large Language Model Approach to Identifying Preoperative Frailty Among Older Adults From Clinical Notes.

Area of Science:

• Artificial Intelligence in Medicine
• Clinical Informatics
• Geriatric Medicine

Background:

• Frailty in patients significantly increases the risk of postoperative mortality and morbidity.
• Identifying frailty is challenging due to its multifactorial nature and reliance on multiple organ systems.
• Accurate frailty phenotyping is crucial for preoperative risk assessment.

Purpose of the Study:

• To develop and validate a large language model (LLM)-based binary classifier for identifying preoperative frailty.
• To utilize unstructured clinical notes for accurate frailty detection in older adults.
• To assess the performance of different LLMs in classifying frailty using distinct phenotyping datasets.

Main Methods:

• Trained and evaluated various LLMs (RoBERTa, BERT, BioBERT, PubMedBERT) on anesthesia preoperative clinic notes.
• Utilized two development datasets: one based on the Vulnerable Elders-13 Survey (VES-13) and another on the electronic frailty index (eFI).
• Assessed model performance using the area under the receiver operating characteristics curve (AUC) on validation sets.

Main Results:

• RoBERTa achieved an AUC of 0.99 on the VES-13 validation set.
• LLMs trained on the eFI dataset demonstrated higher AUCs (0.83-0.87) on the eFI validation set compared to the VES-13 set.
• Models trained on the eFI dataset showed poor discrimination when tested on the VES-13 validation set.

Conclusions:

• Developed and validated an LLM-based classifier to detect preoperative frailty from anesthesia clinical notes.
• LLMs can effectively identify the complex characteristic of frailty using readily available unstructured clinical data.
• This technology offers a promising tool for identifying older adults at risk for frailty in preoperative settings.