BFGTP: A BERT-Guided Two-Stage Molecular Representation Learning Framework for Toxicity Prediction.

Area of Science:

• Computational chemistry and cheminformatics
• Drug discovery and development
• Toxicology and risk assessment

Background:

• Accurate molecular toxicity prediction is crucial for efficient drug development.
• Current methods often rely on fingerprints or graph-based features, with emerging large language models (LLMs) offering new avenues for molecular representation learning.
• Existing LLM approaches for toxicity prediction have limitations, primarily using class embeddings and neglecting sequence embedding information, and could benefit from multi-modal data integration.

Purpose of the Study:

• To propose BFGTP, a novel BERT-guided two-stage molecular representation learning framework for enhanced toxicity prediction.
• To address limitations in current LLM-based molecular representation learning by integrating multi-modal data and sequence information.
• To improve the accuracy and robustness of molecular toxicity prediction for drug development.

Main Methods:

• Developed BFGTP, a framework with independent encoders for three molecular data modalities (fingerprint, sequence, graph).
• Employed dual-level attention mechanisms in the fingerprint encoder for effective multi-category fingerprint integration.
• Utilized a two-stage guidance strategy with contrastive learning for representation fusion and knowledge distillation for value distribution alignment.

Main Results:

• BFGTP demonstrated superior performance across seven toxicity datasets compared to existing baseline methods.
• Achieved the highest Area Under the Curve (AUC) on five datasets and the best average performance across five key evaluation metrics.
• Ablation studies, t-SNE visualization, and case studies validated the effectiveness of BFGTP's components and its capacity for meaningful molecular representation learning.

Conclusions:

• BFGTP effectively integrates multi-modal molecular data and leverages LLMs for improved toxicity prediction.
• The proposed framework offers a significant advancement in molecular representation learning for drug development.
• BFGTP's ability to capture rich molecular information enhances predictive accuracy and provides valuable insights for toxicological assessments.