BioLORD-2023: semantic textual representations fusing large language models and clinical knowledge graph insights.

Area of Science:

• Biomedical Informatics
• Natural Language Processing
• Machine Learning

Background:

• Biomedical knowledge graphs are crucial for semantic modeling.
• Large Language Models (LLMs) offer advanced capabilities for text understanding.
• Integrating LLMs with knowledge graphs can enhance biomedical and clinical semantic models.

Purpose of the Study:

• To investigate the synergistic potential of LLMs and biomedical knowledge graphs.
• To develop a state-of-the-art semantic model for biomedical and clinical domains.
• To improve high-fidelity representations of biomedical concepts and sentences.

Main Methods:

• Utilized the Unified Medical Language System (UMLS) knowledge graph.
• Employed cutting-edge LLMs for representation learning.
• Implemented a three-step approach: contrastive learning, self-distillation, and weight averaging.

Main Results:

• Achieved state-of-the-art results on semantic textual similarity (STS) and biomedical concept representation (BCR).
• Demonstrated significant improvements in clinically named entity linking across 15+ datasets.
• Released a multilingual model supporting 50+ languages, fine-tuned on 7 European languages.

Conclusions:

• BioLORD-2023 offers substantial benefits for clinical pipelines and bioinformatics research.
• The multilingual model democratizes advancements in biomedical semantic representation learning globally.
• BioLORD-2023 is positioned as a valuable tool for future biomedical applications.