Combining accurate tumor genome simulation with crowdsourcing to benchmark somatic structural variant detection

Area of Science:

• Genomics
• Bioinformatics
• Cancer Research

Background:

• Somatic structural variants (SVs) significantly influence cancer cell phenotypes, tumor initiation, aggressiveness, and therapeutic targets.
• Whole-genome sequencing (WGS) of tumors enables comprehensive SV identification for clinical diagnostics and discovering mutagenic mechanisms.
• Existing SV detection algorithms lack systematic benchmarks due to challenges in gold standard creation, resource intensity, and data sharing.

Purpose of the Study:

• To develop a robust simulation framework for evaluating somatic SV detection algorithms.
• To conduct a crowdsourced benchmarking of SV detection methods within the ICGC-TCGA DREAM Somatic Mutation Calling Challenge (SMC-DNA).

Main Methods:

• Extended the BAMSurgeon algorithm to create a simulation framework for somatic SVs.
• Organized a crowdsourced benchmarking challenge (SMC-DNA) involving 15 teams and 204 submissions.
• Evaluated SV detection methods on three distinct tumor datasets.

Main Results:

• Ranked the performance of various SV detection algorithms.
• Identified characteristic error profiles for individual algorithms and general trends across methods.
• Observed that ensemble analysis pipelines did not consistently outperform the best single method.

Conclusions:

• The developed synthetic tumor datasets and SV detection leaderboards serve as a valuable community resource for ongoing algorithm evaluation.
• Findings highlight the need for novel approaches to aggregate SV detection strategies for enhanced accuracy.
• BAMSurgeon is publicly available to support further research in SV detection.