Genomic variant benchmark: if you cannot measure it, you cannot improve it

Area of Science:

• Genomics
• Bioinformatics
• Computational Biology

Background:

• Genomic benchmark datasets are vital for assessing sequencing technologies and bioinformatics tools.
• Current datasets face limitations due to dependence on specific sequencing technologies, reference genomes, and available benchmarking methods.
• Creating comprehensive benchmark datasets is labor-intensive, requiring multiple sequencing technologies, diverse variant calling tools, and extensive manual curation.

Purpose of the Study:

• To review and discuss the utility of existing genomic benchmark datasets.
• To highlight recent advancements in benchmark datasets, particularly those focusing on medically relevant genes and complex genomic regions.
• To underscore the challenges and considerations in developing robust genomic benchmarks.

Main Methods:

• Literature review of existing genomic benchmark datasets.
• Analysis of the strengths and limitations of various benchmarking approaches.
• Focus on recent benchmarks incorporating medically significant genes and complex genomic structures.

Main Results:

• A comprehensive overview of currently available genomic benchmark datasets and their applications.
• Identification of key challenges in benchmark dataset creation, including technological dependencies and manual curation efforts.
• Highlighting the increasing importance of benchmarks for genes with medical relevance and complex genomic characteristics.

Conclusions:

• Genomic benchmark datasets are indispensable for advancing genomics and bioinformatics.
• Future efforts should focus on developing more versatile and comprehensive benchmarks, especially for medically relevant and complex genomic regions.
• Addressing the challenges in dataset creation is key to improving the reliability and applicability of genomic analyses.