Evaluating GPT and BERT models for protein-protein interaction identification in biomedical text

Area of Science:

• Bioinformatics
• Computational Biology
• Natural Language Processing

Background:

• Protein-protein interactions (PPIs) are fundamental to biological processes, disease mechanisms, and drug development.
• The rapid growth of biomedical literature necessitates automated methods for extracting PPI information.
• Pretrained language models (PLMs) show potential for advancing automated information extraction in biology.

Purpose of the Study:

• To evaluate the efficacy of transformer-based language models for detecting protein-protein interactions (PPIs) from biomedical text.
• To compare the performance of models like BioBERT and GPT-4 on established PPI extraction datasets.

Main Methods:

• Evaluated multiple transformer-based models, including BioBERT and GPT-4.
• Tested models on three gold-standard corpora: Learning Language in Logic, Human Protein Reference Database, and Interaction Extraction Performance Assessment.
• Assessed performance using metrics such as precision, recall, and F1 score.

Main Results:

• Bidirectional encoder models generally performed best, with BioBERT achieving high recall (91.95%) and F1 score (86.84%) on the Learning Language in Logic dataset.
• GPT-4 demonstrated competitive performance, achieving the highest precision (88.37%) and a strong F1 score (86.49%) on the same dataset.
• GPT-4's performance indicates its capability in extracting PPIs, even without specific biomedical pre-training.

Conclusions:

• Transformer-based models, including GPT-4, are effective tools for automated protein-protein interaction detection.
• GPT-4 offers a promising approach for mining PPIs from the vast biomedical literature.
• The study highlights the potential of advanced NLP models to accelerate biological research and drug discovery.