Sequential multi-site fine-tuning for incremental deployment of large language models for mobility functional status...

Area of Science:

• Artificial Intelligence
• Clinical Informatics
• Natural Language Processing

Background:

• Large Language Models (LLMs) show promise for clinical information extraction.
• Adapting LLMs across multiple institutions presents challenges in data privacy and resource allocation.
• Extracting functional status, such as mobility, from clinical notes is crucial for patient care.

Purpose of the Study:

• To evaluate sequential multi-site fine-tuning of LLMs for extracting mobility information from clinical notes.
• To compare sequential fine-tuning with centralized joint fine-tuning.
• To assess the generalizability of fine-tuned models across different institutions.

Main Methods:

• A corpus of 600 clinical notes from 3 institutions was used.
• Parameter-efficient low-rank adaptation (LoRA) fine-tuning was applied.
• Sequential fine-tuning involved local training and weight transfer, contrasted with central data pooling for joint fine-tuning.

Main Results:

• Sequential multi-site fine-tuning achieved performance comparable to joint fine-tuning for mobility extraction (F1: 0.861 vs 0.894) and impairment classification (F1: 0.912 vs 0.900).
• Both fine-tuning methods outperformed an untuned 70B model.
• Single-site LoRA models demonstrated robust cross-site generalization.

Conclusions:

• Sequential multi-site fine-tuning is a resource-efficient strategy for adapting LLMs in clinical settings.
• This approach enables local fine-tuning without sharing sensitive patient data.
• It offers a scalable solution for incremental LLM deployment and clinical information extraction from EHRs.