Incorporating entity-level knowledge in pretrained language model for biomedical dense retrieval.

Area of Science:

• Biomedical Natural Language Processing (NLP)
• Information Retrieval
• Machine Learning

Background:

• Pre-trained language models (PLMs) excel in NLP but struggle with biomedical entity ambiguity in dense retrieval.
• Existing PLM-based dense retrieval methods require improvement for the diverse biomedical domain.

Purpose of the Study:

• To enhance dense retrieval performance in the biomedical domain by bridging the semantic gap.
• To enrich query and document representations using external knowledge at the entity level.

Main Methods:

• Incorporated entity-level external knowledge into a BERT-based dense retrieval model.
• Introduced additional self-attention and information interaction modules for fusing text and entity embeddings.
• Developed an entity similarity loss and a weighted entity concatenation mechanism.

Main Results:

• The proposed method (ELK) significantly outperforms state-of-the-art dense retrieval techniques on two biomedical datasets.
• ELK achieved at least a 5% improvement in NDCG metrics compared to coCondenser.
• ELK demonstrated competitive query latency despite its enhanced architecture.

Conclusions:

• Integrating external knowledge effectively addresses the semantic gap in biomedical dense retrieval.
• ELK offers a robust and efficient solution for improving biomedical information retrieval.
• The method shows promise for advancing NLP applications in specialized scientific domains.