For clinical data extraction, QLoRA attains accuracy close to LoRA while requiring lower compute resources

Area of Science:

• Artificial Intelligence in Medicine
• Natural Language Processing (NLP) for Healthcare
• Computational Linguistics

Background:

• Large language models (LLMs) excel at structured data extraction from free text.
• Traditional fine-tuning of LLMs for clinical tasks is computationally expensive and memory-intensive.
• Parameter-Efficient Fine-Tuning (PEFT) offers a solution by updating only a subset of model weights.

Purpose of the Study:

• To evaluate if PEFT methods preserve accuracy on quantized LLMs for clinical data extraction.
• To assess the reduction in memory and GPU requirements using PEFT on quantized models.
• To determine the feasibility of using these methods for hardware-limited research teams.

Main Methods:

• Fine-tuned three Llama-3.1-8B-Instruct variants: non-quantized LoRA and quantized QLoRA (8-bit and 4-bit).
• Utilized the ELMTEX corpus (60,000 clinical summaries) with manual annotations for 15 categories.
• Evaluated models using naïve and advanced prompting, measuring ROUGE, BERTScore, and entity-level F1.

Main Results:

• Fine-tuning significantly outperformed prompting alone, with LoRA improving metrics by 10-20 points.
• QLoRA achieved comparable results, with 8-14 point improvements, only 2-4 points below LoRA.
• Quantization substantially reduced resource needs: QLoRA used fewer GPUs and less peak GPU RAM than LoRA, though training time increased.

Conclusions:

• PEFT on quantized models offers a practical approach for accurate clinical information extraction in resource-constrained environments.
• This method substantially reduces GPU count and memory footprint while retaining most accuracy gains.
• Further research is needed to validate QLoRA across diverse LLM architectures and clinical data types.