geck: trio-based comparative benchmarking of variant calls

Area of Science:

• Genomics
• Bioinformatics
• Statistical genetics

Background:

• Traditional variant calling pipeline accuracy assessment relies on high-confidence truth sets.
• Mendelian constraints offer an alternative, truth set-independent benchmarking approach.
• This method enables evaluation of less-studied variants and diverse populations.

Purpose of the Study:

• To introduce a novel statistical mixture model for comparing variant calling pipelines.
• To enable truth set-independent benchmarking using Mendelian constraints.

Main Methods:

• Developed a statistical mixture model utilizing genotype data from trios.
• Applied the model to compare GATK Unified Genotyper and Haplotype Caller.
• Validated accuracy using high-confidence SNPs from NIST Ashkenazim and Platinum Genome trios.

Main Results:

• The developed model accurately estimates differential precision and recall between pipelines.
• Achieved an uncertainty of 10-3 in precision and recall estimations.
• Demonstrated the model's effectiveness on established trio datasets.

Conclusions:

• Statistical analysis of Mendelian violations provides a robust method for pipeline benchmarking.
• This approach is valuable for evaluating variant callers on diverse populations and challenging genomic regions.
• The geck Python library facilitates the implementation of this novel benchmarking strategy.