Multi-Level Comparison of Machine Learning Classifiers and Their Performance Metrics.

Area of Science:

• Computational chemistry
• Toxicology
• Machine learning

Background:

• Machine learning classification algorithms are crucial for predicting molecular properties like toxicity, offering an alternative to animal testing.
• Assessing model performance is complex, as various metrics can yield conflicting results.

Purpose of the Study:

• To conduct a multi-level comparison of different machine learning classification methods and performance metrics for toxicity prediction.
• To investigate the impact of dataset composition (balanced vs. imbalanced) and classification scenarios (2-class vs. multiclass) on model evaluation.

Main Methods:

• Utilized well-established, standardized protocols for machine learning tasks.
• Applied sum of ranking differences (SRD) and analysis of variance (ANOVA) for robust evaluation.
• Evaluated models on three datasets covering acute and aquatic toxicities.

Main Results:

• Most performance metrics are sensitive to dataset composition, particularly in 2-class classification problems.
• The optimal machine learning algorithm is significantly influenced by the dataset's composition.
• SRD and ANOVA provided robust and sensitive evaluation of model performance.

Conclusions:

• The choice of performance metrics and machine learning algorithms for toxicity prediction must account for dataset characteristics.
• Standardized evaluation protocols are essential for reliable comparison of machine learning models in toxicology.