vcfdist: accurately benchmarking phased small variant calls in human genomes

Area of Science:

• Genomics
• Bioinformatics
• Computational Biology

Background:

• Accurate benchmarking of small variant calling is essential for advancing human whole genome sequencing (WGS) technologies.
• Current evaluation methods exhibit biases related to variant representations, potentially misrepresenting the performance of different variant calling pipelines.

Purpose of the Study:

• To address biases in current variant calling evaluations.
• To propose and validate new methods and tools for more accurate benchmarking.
• To improve the consistency and reliability of performance metrics in variant calling.

Main Methods:

• Exploration of affine gap parameter design for complex variant representation.
• Development and application of the vcfdist tool for evaluation.
• Implementation of local phasing enforcement for accurate assessments.
• Introduction of partial credit for near-correct variant calls.
• Algorithm for clustering dependent variants.
• Utilizing alignment distance metrics alongside precision-recall curves.

Main Results:

• Demonstrated that current evaluations are biased towards specific variant representations.
• vcfdist significantly improves the consistency of insertion and deletion performance metrics.
• Achieved an R-squared value of 0.99996 for vcfdist, compared to 0.97243 for baseline vcfeval.
• Highlighted the importance of local phasing in accurate variant calling evaluation.

Conclusions:

• Current variant calling evaluation methods require refinement to mitigate representation biases.
• vcfdist offers a more robust and consistent approach to benchmarking small variant calling.
• The proposed methods, including partial credit and alignment distance metrics, enhance the understanding of variant caller performance.