Benchmarking algorithms for generalizable single-cell perturbation response prediction.

Area of Science:

• Computational Biology
• Genomics
• Systems Biology

Background:

• Single-cell perturbation technologies offer high-resolution insights into gene function and regulatory networks.
• Large-scale and combinatorial perturbation screens are complex and challenging to perform.
• Computational methods, including foundation models, aim to predict perturbation effects but their efficacy across diverse contexts is uncertain.

Purpose of the Study:

• To comprehensively benchmark 27 computational methods for single-cell perturbation response prediction.
• To systematically assess the generalizability of these methods, including foundation models, across diverse datasets and perturbation scenarios.
• To provide practical guidance for selecting appropriate methods in single-cell research.

Main Methods:

• Evaluation of 27 prediction methods.
• Utilized 29 diverse single-cell perturbation datasets.
• Assessed performance using 6 complementary metrics across multiple scenarios.

Main Results:

• Identified significant variability in method performance and generalizability.
• Emerging foundation models showed promise but also limitations in unseen contexts.
• Current methods often struggle with generalizability across different cellular contexts and perturbation types.

Conclusions:

• Practical guidance for method selection in single-cell perturbation studies is provided.
• The generalizability of perturbation effect prediction remains a key challenge.
• Cellular context embedding approaches are crucial for improving prediction accuracy and robustness.