Spurious correlation inflates performance in single-cell perturbation prediction.

Area of Science:

• Computational biology
• Bioinformatics
• Genomics

Background:

• Computational methods predicting gene expression changes are increasing.
• Current benchmarking relies on correlation or cosine similarity using shared control cells.
• Existing metrics may overestimate method performance due to statistical bias.

Purpose of the Study:

• To identify and address systematic bias in current evaluation metrics for gene expression prediction methods.
• To propose a corrected evaluation approach for computational genomics.

Main Methods:

• Analysis of statistical bias in correlation and cosine similarity metrics.
• Development of a control-splitting procedure to remove bias.
• Reanalysis of published gene expression datasets.

Main Results:

• Standard metrics are systematically inflated by reusing control cell populations.
• Non-informative methods can appear effective, particularly with small control datasets.
• Bias removal significantly reduces previously reported method performance.

Conclusions:

• Current evaluation metrics for gene expression prediction methods are unreliable.
• A simple control-splitting procedure corrects for bias and provides a more accurate assessment.
• This highlights the need for rigorous, unbiased evaluation in computational biology.