A benchmark study of simulation methods for single-cell RNA sequencing data.

Area of Science:

• Computational biology
• Genomics
• Bioinformatics

Background:

• Single-cell RNA sequencing (scRNA-seq) data simulation is essential for assessing computational analysis tools, particularly when experimental ground truth is unavailable.
• The accuracy of scRNA-seq data evaluation hinges on how well simulation methods replicate experimental data properties.
• Despite numerous proposed simulation methods, a systematic evaluation is lacking.

Purpose of the Study:

• To develop a comprehensive framework, SimBench, for evaluating scRNA-seq data simulation methods.
• To benchmark existing simulation techniques using a standardized approach and diverse datasets.
• To identify strengths and weaknesses of current simulation methods for guiding future research.

Main Methods:

• Development of SimBench, a comprehensive evaluation framework for scRNA-seq simulation.
• Inclusion of a kernel density estimation measure for benchmarking.
• Systematic evaluation of 12 simulation methods across 35 scRNA-seq experimental datasets, assessing data properties, biological signal preservation, scalability, and applicability.

Main Results:

• The benchmark revealed significant performance variations among the 12 evaluated scRNA-seq simulation methods.
• Simulating specific data characteristics presented varying levels of difficulty for the methods.
• Limitations were identified, notably in maintaining the heterogeneity of data distribution.

Conclusions:

• The SimBench framework provides a standardized approach to evaluate and compare scRNA-seq data simulation methods.
• The study highlights the need for improved simulation techniques, particularly in capturing complex data distributions.
• Publicly available R packages for the framework and datasets will aid researchers in selecting appropriate simulation methods and advancing the field.