Structure-activity relationship-based chemical classification of highly imbalanced Tox21 datasets

Area of Science:

• Computational Toxicology
• Cheminformatics
• Machine Learning

Background:

• Toxicity datasets are often imbalanced due to specific toxicant-target interactions, hindering Structure-Activity Relationship (SAR) based chemical classification.
• Traditional imbalance handling techniques like undersampling and oversampling have limitations, including information loss or the introduction of artificial, overlapping data points.

Purpose of the Study:

• To enhance prediction accuracy in imbalanced chemical toxicity classification.
• To evaluate the effectiveness of SMOTEENN, a hybrid data rebalancing technique, in improving SAR-based classification performance.

Main Methods:

• Employed SMOTEENN (Synthetic Minority Over-sampling Technique with Edited Nearest Neighbor) to address class imbalance in the Tox21 dataset.
• Utilized Random Forest (RF) as the base classifier with bagging, comparing four methods: RF, RF with Random Undersampling (RUS), RF with SMOTE (SMO), and RF with SMOTEENN (SMN).
• Assessed performance using nine metrics, focusing on F1 score, Matthews correlation coefficient, and Brier score for consistent evaluation across 12 in vitro bioassays.

Main Results:

• The SMOTEENN-based method (SMN) demonstrated statistically significant superior performance compared to RF, RUS, and SMO.
• A strong negative correlation was observed between prediction accuracy and the imbalance ratio (IR).
• SMN's effectiveness decreased when the imbalance ratio exceeded approximately 28.

Conclusions:

• SMOTEENN is a highly effective strategy for improving SAR-based imbalanced chemical toxicity classification.
• Data rebalancing techniques are crucial for accurate computational toxicology predictions, particularly in identifying active compounds within large inactive datasets.
• The study highlights the importance of choosing appropriate imbalance handling methods, as performance is sensitive to the degree of data skewness.