Adaptive, Privacy-Preserving Small Language Models for Multi-Task Clinical Assistance

Area of Science:

• Artificial Intelligence in Medicine
• Natural Language Processing
• Clinical Informatics

Background:

• Large language models (LLMs) require significant resources and managing multiple task-specific models is inefficient for clinical settings.
• Developing tailored, privacy-preserving, and deployable language models is crucial for healthcare AI.
• Small language models (SLMs) offer a potential alternative for efficient and customizable clinical AI solutions.

Purpose of the Study:

• To evaluate if a single, fine-tuned SLM can match or exceed LLM performance across various clinical tasks.
• To enable hospitals to deploy efficient, privacy-preserving language models without managing multiple systems.
• To assess the feasibility of using a multi-task SLM for diverse clinical applications.

Main Methods:

• Fine-tuning of varying-sized SLMs using low-rank adaptation (LoRA) on clinical datasets.
• Evaluation across three tasks: medical report labeling, DICOM series description harmonization, and impression generation.
• Comparison of single-task SLMs, a multi-task SLM, and GPT-4o using zero-shot and few-shot prompting.

Main Results:

• The multi-task SLM achieved superior performance: F1 score of 0.894 in labeling (vs. GPT-4o's 0.728) and 0.975 accuracy in harmonization (vs. GPT-4o's 0.878).
• Impression generation showed a higher Likert score for the multi-task SLM (4.39 ± 1.00) compared to GPT-4o (3.65 ± 1.00).
• OPT-350m was identified as the optimal SLM for this multi-task approach.

Conclusions:

• A single fine-tuned SLM can function as a general-purpose clinical assistant, matching or surpassing larger models.
• This approach offers lower resource requirements, enhanced customizability, and privacy protection for clinical AI.
• Fine-tuning one SLM for multiple clinical tasks addresses practical deployment demands in diverse healthcare settings.