TabNet and TabTransformer: Novel Deep Learning Models for Chemical Toxicity Prediction in Comparison With Machine...

Area of Science:

• Computational chemistry and toxicology
• Machine learning in drug discovery
• Bioinformatics and cheminformatics

Background:

• Accurate chemical toxicity prediction is vital for drug discovery, environmental safety, and regulatory compliance.
• Traditional machine learning methods and feature selection techniques have limitations in capturing complex molecular interactions for toxicity prediction.
• The need for robust, interpretable, and efficient models for assessing chemical safety is increasing.

Purpose of the Study:

• To evaluate and compare the performance of advanced deep learning architectures (TabNet, TabTransformer) against traditional machine learning models for predicting chemical toxicity across 12 endpoints.
• To assess the interpretability of deep learning models using SHAP analysis, particularly under class imbalance conditions.
• To determine the suitability of these models as alternatives to traditional in vitro and in vivo toxicity testing methods.

Main Methods:

• Utilized a dataset of 12,228 training and 3,057 test samples with 801 molecular descriptors.
• Implemented traditional models (XGBoost, CatBoost, SVM, voting classifier) with feature selection (PCA, RFE, MI).
• Trained TabNet and TabTransformer directly on the full feature set, assessing performance with metrics like AUC-ROC, F1-score, AUPR, and MCC, complemented by SHAP analysis.

Main Results:

• TabNet and TabTransformer consistently outperformed traditional classifiers, achieving up to 96% AUC-ROC for endpoints like SR.ARE and SR.p53.
• TabTransformer demonstrated superior performance on complex labels due to self-attention mechanisms, while TabNet offered efficient dynamic feature selection.
• SHAP analysis identified key molecular descriptors (e.g., VSAEstate6, MoRSEE8), enhancing model transparency and interpretability.

Conclusions:

• TabNet and TabTransformer represent a significant advancement in chemical toxicity prediction, offering superior accuracy and generalizability over traditional methods.
• The interpretability provided by SHAP analysis ensures these deep learning models are transparent and trustworthy for regulatory applications.
• These advanced models provide a promising, cost-effective, and ethical alternative to conventional in vitro and in vivo toxicity assessment approaches.