Fine-Tuning Methods for Large Language Models in Clinical Medicine by Supervised Fine-Tuning and Direct Preference...

Area of Science:

• Artificial Intelligence
• Medical Informatics
• Natural Language Processing

Background:

• Large language models (LLMs) show potential in medicine but often require fine-tuning for optimal performance.
• Supervised fine-tuning (SFT) and direct preference optimization (DPO) are key LLM fine-tuning techniques.
• Guidance on selecting between SFT and DPO in clinical settings is limited.

Purpose of the Study:

• To compare the effectiveness of SFT and DPO for various medical natural language tasks.
• To provide data-driven recommendations for clinical informaticists on choosing fine-tuning methods.
• To inform the development and deployment of LLMs in healthcare operations.

Main Methods:

• Utilized Llama3 8B and Mistral 7B v2 models for comparison.
• Evaluated SFT and DPO performance on four core medical NLP tasks: text classification, clinical reasoning, summarization, and triage.
• Quantified performance using accuracy, Likert scales, and F1-scores, alongside computational resource analysis.

Main Results:

• DPO significantly improved clinical reasoning (accuracy from 22% to 40%) and summarization (Likert scale from 3.93 to 4.08) compared to SFT.
• SFT was sufficient for text classification (F1-score up to 0.98), while DPO showed mixed results.
• DPO fine-tuning demanded 2-3 times more computational resources than SFT alone.

Conclusions:

• SFT is adequate for straightforward tasks like text classification.
• DPO, applied after SFT, enhances performance on complex tasks including triage and clinical reasoning.
• Findings guide the strategic deployment of LLM fine-tuning in medical applications.