How (Not) to Generate a Highly Predictive Biomarker Panel Using Machine Learning.

Area of Science:

• Proteomics
• Bioinformatics
• Machine Learning

Background:

• Biomarker discovery in proteomics often employs machine learning.
• Feature selection is a critical step in building predictive models.
• Inappropriate feature selection can lead to overestimated model performance.

Purpose of the Study:

• To demonstrate a common data processing error in proteomics biomarker studies.
• To illustrate how biased feature selection inflates machine learning model accuracy.
• To provide guidance on applying machine learning to proteomics data correctly.

Main Methods:

• Demonstration of a flawed feature selection strategy.
• Building a classification model using biased feature selection.
• Simulating a dataset to highlight the impact of data leakage.

Main Results:

• An artificially high classification accuracy of 92% and AUC of 0.98 was achieved.
• The inflated performance was demonstrated on a dataset relying on random numbers.
• The study identified test data leakage into the feature selection step as the core issue.

Conclusions:

• Biomarker panels generated by selecting features across all data before cross-validation are unreliable.
• Test data leakage during feature selection is a common pitfall in machine learning for proteomics.
• Correct application of machine learning requires careful separation of feature selection and model validation to prevent inflated accuracies.