Computational Study Protocol: Leveraging Synthetic Data to Validate a Benchmark Study for Differential Abundance...

Area of Science:

• Microbiome research
• Computational biology
• Statistical analysis

Background:

• Synthetic data utility in benchmark studies hinges on mimicking real-world conditions and experimental data reproducibility.
• Nearing et al. (1) assessed 14 differential abundance tests on 38 experimental 16S rRNA datasets.
• This study generates synthetic datasets to mimic experimental data for verifying Nearing et al.'s findings.

Purpose of the Study:

• To rigorously assess the similarity between synthetic and experimental data using statistical tests.
• To validate the conclusions drawn by Nearing et al. (1) regarding differential abundance test performance.
• To adhere to SPIRIT guidelines for robust, transparent, and unbiased study planning.

Main Methods:

• Replication of Nearing et al.'s (1) methodology using synthetic data simulated by two distinct tools.
• Comparison of 38 experimental datasets with synthetic counterparts using equivalence tests on 46 data characteristics and principal component analysis.
• Application of 14 differential abundance tests to both synthetic and experimental datasets to evaluate consistency in significant feature identification.

Main Results:

• Equivalence tests and principal component analysis will assess data similarity.
• Differential abundance tests will be evaluated for consistency in identifying significant features.
• Correlation analysis and multiple regression will explore the impact of data characteristic differences on test results.

Conclusions:

• Synthetic data enables validation of findings through controlled experiments.
• This study assesses synthetic data's replication accuracy and validates recent differential abundance methods.
• This is a pioneering computational benchmark study incorporating synthetic data for differential abundance method validation, adhering to SPIRIT guidelines for enhanced transparency and reproducibility.