LBERT: Lexically aware Transformer-based Bidirectional Encoder Representation model for learning universal bio-entity...

Area of Science:

• Biomedical Natural Language Processing
• Bioinformatics
• Computational Biology

Background:

• Advanced Natural Language Processing (NLP) is crucial for managing and structuring the growing volume of biomedical data.
• Bio-Entity Relation Extraction (BRE) is vital for knowledge discovery in the biomedical domain.
• Current deep learning models for BRE often lack task universality and fail to incorporate local syntactic context.

Purpose of the Study:

• To propose a universal Bio-Entity Relation Extraction (BRE) model, LBERT (Lexically aware Transformer-based Bidirectional Encoder Representation), that captures both local and global contextual information.
• To evaluate LBERT's performance across various biomedical relation types, including protein-protein interaction (PPI), drug-drug interaction, and protein-bio-entity relations.
• To demonstrate the effectiveness of lexical features and distance-adjusted attention in enhancing BRE predictive accuracy.

Main Methods:

• Development of LBERT, a novel Lexically aware Transformer-based Bidirectional Encoder Representation model.
• Exploration of both local syntactic and global semantic contexts for sentence-level classification tasks.
• Comparative analysis against state-of-the-art deep learning models on multiple BRE tasks.

Main Results:

• LBERT significantly outperforms existing deep learning models in protein-protein interaction (PPI), drug-drug interaction, and protein-bio-entity relation classification.
• LBERT demonstrates statistically significant improvements over BioBERT in detecting bio-entity relations within large corpora like PPI.
• Ablation studies confirm the contribution of lexical features and distance-adjusted attention to improved prediction performance.

Conclusions:

• LBERT offers a universal and effective approach to Bio-Entity Relation Extraction by integrating lexical awareness and multi-contextual representations.
• The model's superior performance highlights the importance of incorporating local syntactic information alongside global context in biomedical NLP.
• LBERT represents a significant advancement in structuring biomedical knowledge and facilitating data-driven discoveries.