Predicting Drug-Side Effect Relationships From Parametric Knowledge Embedded in Biomedical BERT Models:...

Area of Science:

• Biomedical Informatics
• Natural Language Processing
• Pharmacovigilance

Background:

• Adverse drug reactions (ADRs) present significant patient health risks.
• Traditional word embedding models like Word2Vec struggle with biomedical text specificity.
• BERT models show promise but require further research for predicting unknown drug-side effect relationships.

Purpose of the Study:

• To predict novel drug-side effect relationships using parametric knowledge from biomedical BERT models.
• To leverage embedding vector similarities of known relationships for enhanced prediction.
• To identify potential drug-side effect candidates with unknown causal mechanisms.

Main Methods:

• Utilized 158,096 drug-side effect pairs from the SIDER database to create an adjacency matrix.
• Calculated cosine similarity between drug and side effect word embedding vectors.
• Computed relation scores for 8,235,435 drug-side effect pairs and evaluated using Area Under the Curve (AUC).

Main Results:

• The clagator/biobert_v1.1 model achieved an AUC of 0.915, surpassing Word2Vec (AUC 0.848).
• BERT models pretrained on biomedical corpora outperformed vanilla BERT (AUC 0.857).
• External validation with FDA Adverse Event Reporting System data showed high statistical significance (P<.001, OR=4.822).

Conclusions:

• A novel method for predicting drug-side effect relationships using pretrained biomedical language models was developed.
• BERT-based models excel in predicting drug-side effect relationships due to contextual understanding.
• The approach demonstrates practical utility for predicting and managing ADRs.