BioInstruct: instruction tuning of large language models for biomedical natural language processing

Area of Science:

• Biomedical Natural Language Processing (BioNLP)
• Artificial Intelligence in Medicine

Background:

• Large language models (LLMs) show promise in biomedical natural language processing (BioNLP).
• Domain-specific fine-tuning is crucial for optimizing LLM performance in specialized fields like medicine.

Purpose of the Study:

• To introduce the BioInstruct dataset for instruction-tuning LLMs.
• To evaluate the impact of domain-specific instruction tuning on BioNLP tasks.
• To explore the synergy between instruction tuning and multi-task learning principles.

Main Methods:

• Developed BioInstruct, a dataset of 25,005 instructions for LLMs (LLaMA 1 and 2).
• Utilized GPT-4 to generate instructions based on human-curated samples.
• Employed Low-Rank Adaptation (LoRA) for efficient fine-tuning.
• Evaluated instruction-tuned LLMs on question answering (QA), information extraction (IE), and text generation (GEN) tasks.

Main Results:

• Instruction-tuned LLMs achieved significant performance gains: 17.3% in QA accuracy, 5.7% in IE F1 score, and 96% in GEN GPT-4 score.
• The 7B-parameter instruction-tuned LLaMA 1 model demonstrated competitive or superior performance compared to other domain-specific LLMs.
• Performance improvements were notably higher when instruction fine-tuning involved closely related tasks, indicating multi-task learning synergies.

Conclusions:

• The BioInstruct dataset is a valuable resource for advancing BioNLP.
• Instruction-tuned LLMs represent the state-of-the-art for high-performing BioNLP applications.
• Synergies between instruction tuning and multi-task learning enhance LLM capabilities in the biomedical domain.