Classifiers and their Metrics Quantified.

Area of Science:

• Computational chemistry
• Cheminformatics
• Bioinformatics

Background:

• Classification models are widely used in molecular modeling for predicting binary outcomes like bioactivity or protein interactions.
• Common evaluation metrics (e.g., accuracy, true positive rate) may overestimate model performance on real-world prospective datasets.
• Retrospective or artificially generated datasets can lead to misleading performance assessments.

Purpose of the Study:

• To address the overestimation of predictive model performance in molecular modeling.
• To propose a novel method for analyzing metric performance based on data balance.
• To provide guidance on selecting appropriate evaluation metrics for study design.

Main Methods:

• Systematic analysis of metric value surface generation influenced by data balance.
• Development and application of an inverse cumulative distribution function over metric surfaces.
• Theoretical analysis complemented by a practical chemogenomic virtual screening example.

Main Results:

• Demonstrated how data balance influences metric value surfaces.
• Introduced a distribution analysis method for evaluating classification model performance.
• Highlighted the critical importance of careful metric selection and interpretation in virtual screening.

Conclusions:

• Standard performance metrics can be unreliable for prospective predictions.
• The proposed distribution analysis offers a more robust approach to metric evaluation.
• Informed metric selection is crucial for reliable molecular modeling and virtual screening outcomes.