Addressing Imbalanced Classification Problems in Drug Discovery and Development Using Random Forest, Support Vector...

Area of Science:

• Machine Learning
• Data Science
• Computational Chemistry

Background:

• Class imbalanced datasets pose challenges for classification models, often leading to poor minority class prediction.
• Existing techniques to address class imbalance include data-level, algorithm-level, and hybrid methods.
• A comprehensive analysis of these techniques' performance across varying class ratios is lacking.

Purpose of the Study:

• To conduct an in-depth analysis of data balancing techniques for imbalanced classification.
• To evaluate the performance of different techniques against varying class ratios using machine learning and AutoML tools.
• To compare the effectiveness of threshold optimization, internal balancing, and data balancing methods like SMOTETomek.

Main Methods:

• Generated 27 datasets with nine class ratios from three drug discovery datasets.
• Employed Random Forest (RF) and Support Vector Machine (SVM) as machine learning classifiers.
• Utilized AutoGluon-Tabular and H2O AutoML as representative AutoML tools.
• Evaluated techniques including threshold optimization (GHOST, AUPR), class-weighting, and SMOTETomek.

Main Results:

• Threshold optimization did not impact ranking metrics (AUC, AUPR), but class-weighting and SMOTETomek did.
• Machine learning methods (RF, SVM) and AutoML tools showed significant improvements in F1 score, MCC, and balanced accuracy.
• Performance improvements generally increased as class ratios decreased, with peak F1 and MCC scores at a 0.3 ratio.
• Combined external balancing techniques outperformed internal methods, and AutoML tools performed comparably or better than ML models.

Conclusions:

• No single data balancing technique consistently outperformed others across all datasets.
• Combining multiple external balancing techniques is recommended for optimal imbalanced classification.
• AutoML tools offer competitive or superior performance compared to traditional ML models when handling imbalanced data with appropriate techniques.