How benchmarking of bioinformatics tools is essential for informed workflow selection: a case study on SARS-CoV-2...

Selecting appropriate bioinformatics tools is critical for accurate and reproducible analysis, particularly in support of genomic surveillance and molecular biomarker monitoring. The importance of these analyses is underscored by the need for effective public health responses to emerging diseases like SARS-CoV-2.

By using the detection of SARS-CoV-2 subgenomic RNAs (sgRNAs) as a case study, we show the importance of systematic benchmarking in selecting optimal workflows. We generated 25 synthetic Illumina datasets simulating both shotgun and amplicon sequencing strategies, along with a real-world wastewater dataset. Using these datasets, we assessed the influence of key variables including mutation profiles, read lengths, aligner choice, and primer design for targeted sequencing.

Our results revealed substantial performance variability: common tools developed to identify sgRNAs struggled with shotgun data and were sensitive to mutations depending on the chosen aligner, while amplicon sequencing improved detection sensitivity, with aligners and primer design choices still significantly impacting outcomes.

Our results highlight the need for benchmarking steps and analyses to inform workflow selection. Without such evaluations, researchers risk drawing inaccurate conclusions from suboptimal workflows. This case study underscores the value of context-aware tool selection and encourages standardised benchmarking practices to ensure reproducibility and reliability in bioinformatics analysis, particularly in evidence-based decision-making environments such as public health and policymaking.