Discriminative Index: A Novel Indicator for Evaluating Machine Learning Algorithms in Laboratory Medicine

Area of Science:

• Biomedical informatics
• Machine learning
• Clinical decision support

Background:

• Machine learning models are vital in clinical practice.
• Traditional metrics (sensitivity, specificity, accuracy) miss predictive distribution's structural details.
• This limits optimizing decision thresholds and clinical support.

Purpose of the Study:

• Introduce the D-index, a novel metric for quantifying asymmetry in predicted probability distributions.
• Evaluate the D-index's sensitivity to distributional characteristics and its practical utility.
• Demonstrate the D-index's value in enhancing clinical decision-making for machine learning models.

Main Methods:

• Proposed the D-index: measuring skewness difference between positive and negative predictions.
• Validated using simulated datasets with varying skewness and kurtosis.
• Tested on real-world MALDI-TOF mass spectrometry data for vancomycin resistance prediction, comparing deep learning, XGBoost, and random forest.

Main Results:

• D-index selectively increased with performance in skewness-dominant simulated distributions.
• Real-world data showed D-index differentiating models with similar conventional performance but different distributional structures.
• Higher D-index models retained more confident classifications and improved accuracy under gray-zone strategies.

Conclusions:

• The D-index offers structural insights beyond traditional metrics.
• It identifies models suitable for gray-zone strategies, enhancing clinical decision-making.
• This distribution-aware approach improves safety, efficiency, and utility of biomedical machine learning models.