VitroBert: modeling DILI by pretraining BERT on in vitro data.

Area of Science:

• Computational chemistry
• Toxicology
• Machine learning

Background:

• Drug-induced liver injury (DILI) poses challenges due to complexity and data limitations.
• Unsupervised molecular representation learning often misses biological interaction insights.
• Existing models struggle with severe class imbalance in DILI datasets.

Purpose of the Study:

• Introduce VitroBERT, a BERT model pretrained on in vitro assay data for biologically informed molecular embeddings.
• Evaluate VitroBERT's performance in predicting in vivo DILI endpoints.
• Identify optimal loss functions for imbalanced DILI prediction tasks.

Main Methods:

• Pretrained a bidirectional encoder representations from transformers (BERT) model (VitroBERT) on large-scale in vitro assay profiles.
• Compared VitroBERT embeddings against unsupervised pretraining (MolBERT) for DILI endpoint prediction.
• Assessed various loss functions (BCE, weighted BCE, Focal loss, weighted Focal loss) for class imbalance.

Main Results:

• VitroBERT embeddings improved biochemistry (29%) and histopathology (16%) DILI endpoint predictions over MolBERT.
• No significant improvement was observed for clinical DILI prediction tasks.
• Weighted Focal loss demonstrated superior performance in handling class imbalance for DILI tasks.

Conclusions:

• Integrating biological context via in vitro data into molecular models enhances DILI prediction.
• Appropriate loss function selection is crucial for improving performance on imbalanced DILI datasets.
• VitroBERT offers a promising approach for biologically informed molecular representation learning in toxicology.