How doppelgänger effects in biomedical data confound machine learning.

Area of Science:

• Biomedical data science
• Computational drug discovery
• Machine learning in pharmacology

Background:

• Machine learning (ML) models accelerate drug target identification.
• Cross-validation is standard for evaluating ML models in drug development.
• Data doppelgängers, or highly similar independent datasets, can compromise validation reliability.

Purpose of the Study:

• To investigate the prevalence and impact of data doppelgängers in biomedical data.
• To demonstrate how data doppelgängers arise and affect ML model evaluation.
• To propose strategies for mitigating the doppelgänger effect in drug development.

Main Methods:

• Analysis of biomedical datasets to identify data doppelgängers.
• Simulations to demonstrate the origin and effects of data doppelgängers.
• Evaluation of ML model performance with and without accounting for data doppelgängers.

Main Results:

• Data doppelgängers are prevalent in biomedical datasets.
• The doppelgänger effect leads to overestimated ML model performance.
• Failure to account for doppelgängers results in unreliable model validation.

Conclusions:

• Data doppelgängers pose a significant, uncharacterized challenge in ML-driven drug discovery.
• Proactive identification of data doppelgängers before training-validation splits is essential.
• Addressing the doppelgänger effect ensures more robust and reliable ML model development for identifying drug targets.