Leveraging pre-trained language models for mining microbiome-disease relationships.

Area of Science:

• Computational Biology
• Bioinformatics
• Natural Language Processing

Background:

• The human microbiome significantly impacts health, but microbe-disease links are fragmented in literature.
• Structured extraction of microbe-disease interactions is crucial for advancing research.
• Deep learning and NLP advancements offer new avenues for information extraction.

Purpose of the Study:

• To leverage state-of-the-art deep learning language models for extracting microbe-disease relationships.
• To evaluate and fine-tune language models for domain-specific biomedical text analysis.

Main Methods:

• Evaluated multiple pre-trained large language models (LLMs) in zero-shot and few-shot settings.
• Fine-tuned LLMs including GPT-3, BioGPT, BioMedLM, BERT, BioMegatron, PubMedBERT, BioClinicalBERT, and BioLinkBERT.
• Assessed model performance using labeled training data for microbe-disease interaction extraction.

Main Results:

• Out-of-the-box LLMs performed poorly, highlighting the need for domain-specific fine-tuning.
• Fine-tuned models, particularly GPT-3, BioMedLM, and BioLinkBERT, achieved state-of-the-art performance.
• Achieved average F1 scores, precision, and recall exceeding previous benchmarks.

Conclusions:

• Pre-trained language models are effective transfer learners when fine-tuned with domain-specific data.
• Fine-tuning enables state-of-the-art extraction of microbiome-disease interactions with limited data.
• This approach enhances the accessibility of crucial information from biomedical literature.