Mobility functional status ascertainment in electronic health records using large language models.

Area of Science:

• Medical Informatics
• Artificial Intelligence in Healthcare
• Geriatric Medicine

Background:

• Assessing functional status, particularly mobility, is crucial for precision medicine in an aging global population.
• Electronic Health Records (EHRs) contain valuable unstructured data on patient mobility.
• Standardizing mobility status from clinical notes is challenging but essential for clinical applications and research.

Purpose of the Study:

• To investigate the efficacy of Large Language Models (LLMs) in extracting and standardizing patient mobility status from unstructured EHR data.
• To evaluate different LLM prompting strategies for mobility data extraction and impairment classification.
• To assess the clinical plausibility and generalizability of LLM-derived mobility assessments.

Main Methods:

• Annotated 600 clinical notes from three healthcare institutions for mobility expressions and impairments.
• Utilized the open-source Llama 3 model with zero-shot, few-shot, and task decomposition prompting.
• Performed error analysis and secondary analysis considering "clinically reasonable inferences" as correct.
• Implemented a local, deterministic setup for enhanced trustworthiness and privacy.

Main Results:

• The final LLM model achieved high performance: F1-scores of 0.876 for mobility extraction and 0.897 for impairment classification.
• Considering clinically reasonable inferences, F1-scores increased to 0.962 and 0.948, respectively.
• The local, deterministic setup ensured consistent outputs, protected privacy, and demonstrated cross-institution generalizability.

Conclusions:

• LLM-based solutions are feasible for extracting and standardizing mobility functional status from unstructured EHR data.
• This technology can significantly support precision medicine, clinical decision-making, and healthcare research.
• The findings underscore the potential of AI to unlock valuable insights from complex clinical text.