A large-scale benchmark for network inference from single-cell perturbation data.

Area of Science:

• Computational Biology
• Systems Biology
• Genomics

Background:

• Mapping biological mechanisms is crucial for drug discovery and hypothesis generation.
• High-throughput single-cell gene expression data enables large-scale causal gene-gene interaction inference.
• Evaluating network inference methods is challenging due to lack of ground-truth and limitations of synthetic data.

Purpose of the Study:

• Introduce CausalBench, a novel benchmark suite for evaluating network inference methods.
• Provide a realistic assessment of causal inference performance using real-world, large-scale single-cell perturbation data.
• Facilitate the development of advanced network inference methods for computational biology and drug discovery.

Main Methods:

• Developed CausalBench, a benchmark suite utilizing real-world, large-scale single-cell perturbation data.
• Incorporated biologically-motivated metrics and distribution-based interventional measures for enhanced evaluation.
• Conducted a systematic evaluation of state-of-the-art causal inference methods on the CausalBench suite.

Main Results:

• Existing causal inference methods exhibit poor scalability, limiting their performance on real-world data.
• Methods using interventional data did not outperform those using only observational data, contrary to synthetic benchmark results.
• CausalBench highlights performance discrepancies between real-world data and synthetic benchmarks for network inference.

Conclusions:

• CausalBench offers a transformative tool for computational biology, bridging theoretical innovation and practical application in drug discovery.
• The benchmark enables the development of improved causal network inference methods through community challenges.
• Provides a reliable framework for practitioners to track progress in network inference methods for real-world interventional data.