BioTriplex: a full-text annotated corpus for fine-tuning language models in gene-disease relation extraction tasks

Area of Science:

• Biomedical informatics
• Computational biology
• Natural Language Processing

Background:

• Machine learning for biomedical text analysis requires specialized models due to technical terminology and semantic variations.
• Existing large language models (LLMs) struggle with the nuances of biomedical literature.
• There is a need for annotated full-text datasets to fine-tune LLMs for specific biomedical applications, particularly gene-disease relationship extraction.

Purpose of the Study:

• To introduce BioTriplex, a novel annotated corpus of full-length biomedical research articles.
• To develop and evaluate a fine-tuned language model for extracting gene-disease relationships.
• To demonstrate the effectiveness of specialized datasets in improving LLM performance for biomedical tasks.

Main Methods:

• Developed BioTriplex, a corpus of 100 full-length biomedical articles manually annotated with diseases, genes, and 21 subtypes of gene-disease relationships.
• Utilized BioTriplex to fine-tune the LLaMA 3.1 8B language model for gene-disease relation extraction.
• Compared the performance of the fine-tuned model against zero-shot and few-shot approaches within the same architecture and against other state-of-the-art LLMs (GPT-4, Claude Sonnet 3.7).

Main Results:

• The BioTriplex-trained LLaMA 3.1 8B model significantly outperformed baseline and other advanced LLMs in gene-disease relation extraction.
• The fine-tuned model demonstrated superior accuracy and a broader, more granular classification of gene-disease relationship types.
• BioTriplex proved to be a valuable resource for enhancing LLM capabilities in biomedical information extraction.

Conclusions:

• BioTriplex is an effective resource for fine-tuning language models for biomedical applications.
• Specialized datasets are crucial for improving the performance of LLMs in complex scientific domains like gene-disease relationship extraction.
• The fine-tuned LLaMA 3.1 8B model represents a significant advancement in automated extraction of gene-disease relationships.