Optimizing large language models for ontology-based annotation: a study on gene ontology in biomedical texts

Area of Science:

• Biomedical Informatics
• Computational Biology
• Natural Language Processing

Background:

• Automated ontology annotation is crucial for knowledge management in biology and biomedicine.
• Traditional models like RNNs and Bi-GRUs face limitations with complex biomedical terms.

Purpose of the Study:

• To evaluate large language models (LLMs) for Gene Ontology (GO) annotation.
• To compare LLM performance against traditional models using the CRAFT dataset.

Main Methods:

• Fine-tuning LLMs (MPT-7B, Phi, BiomedLM, Meditron) on the CRAFT dataset.
• Assessing performance via F1 score, semantic similarity, memory usage, and inference speed.
• Exploring parameter-efficient fine-tuning (PEFT) and advanced prompting.

Main Results:

• LLMs demonstrate competitive accuracy and qualitatively higher semantic consistency for complex terms.
• Bi-GRU baselines remain strong in raw accuracy.
• LLMs exhibit high resource requirements, impacting computational efficiency.

Conclusions:

• LLMs offer complementary strengths to traditional methods for biomedical ontology annotation.
• Parameter-efficient fine-tuning (PEFT) can mitigate resource demands.
• Further optimization and domain-specific training are needed for practical LLM deployment.