Sequential multi-site fine-tuning for incremental deployment of large language models for mobility functional status...

This study evaluated sequential multi-site fine-tuning of large language models (LLMs), simulating incremental deployment across institutions for extracting mobility functional status from unstructured clinical notes.

We assembled a corpus of 600 clinical notes from 3 institutions, yielding 3810 annotated note sections (1200 training; 2610 testing). Building upon our previously established LLM mobility extraction pipeline, which utilized optimized prompt engineering, we applied parameter-efficient low-rank adaptation (LoRA) fine-tuning. We evaluated sequential multi-site fine-tuning, where the model is trained locally and only the updated LoRA weights are transferred to the next site. We compared this against joint fine-tuning, which requires pooling all multi-site patient data centrally before training. Performance was measured by micro F1-scores for 2 distinct subtasks: mobility extraction and impairment classification.

Sequential multi-site fine-tuning yielded comparable results to centralized joint fine-tuning for mobility extraction (F1, 0.861 vs 0.894) and impairment classification (F1, 0.912 vs 0.900). Both sequential and joint fine-tuning outperformed an untuned 70B model. Additionally, single-site LoRA models showed robust cross-site generalization.

Sequential multi-site fine-tuning offers a resource-friendly approach to adapt LLMs to clinical environments. By retaining previously acquired knowledge without severe interference, this strategy allows for local fine-tuning at each institution without the need to pool or share sensitive patient data required for joint fine-tuning.

Sequential multi-site fine-tuning provides a viable solution for the scalable, incremental deployment of LLMs across institutions, showing promising generalizability for clinical information extraction from electronic health record narratives.