BMRetriever: Tuning Large Language Models as Better Biomedical Text Retrievers

Area of Science:

• Biomedical Informatics
• Information Retrieval
• Natural Language Processing

Background:

• Effective biomedical retrieval models are crucial for knowledge-intensive tasks.
• Challenges include limited annotated data and computational resources.

Purpose of the Study:

• To develop BMRetriever, a series of dense retrievers for improved biomedical information retrieval.
• To address data scarcity and computational limitations in the field.

Main Methods:

• Unsupervised pre-training on large biomedical corpora.
• Instruction fine-tuning using labeled datasets and synthetic data pairs.
• Development of parameter-efficient dense retriever variants (410M and 2B parameters).

Main Results:

• BMRetriever demonstrated efficacy across 5 biomedical tasks and 11 datasets.
• The 410M variant outperformed significantly larger baselines (up to 11.7x).
• The 2B variant achieved performance comparable to models over 5B parameters.

Conclusions:

• BMRetriever offers a powerful and efficient solution for biomedical retrieval.
• The released model checkpoints and data promote transparency and reproducibility.
• BMRetriever can be applied to new biomedical domains and tasks.