Comprehensive evaluation of ACMG/AMP-based variant classification tools

Area of Science:

• Genomic Medicine
• Clinical Bioinformatics
• Computational Biology

Background:

• The American College of Medical Genetics and Genomics/Association for Molecular Pathology (ACMG/AMP) guidelines are standard for clinical variant interpretation.
• A lack of comparative studies on ACMG/AMP software tools hinders clinical adoption.
• Clinicians need evidence-based guidance for selecting variant interpretation tools.

Purpose of the Study:

• To benchmark and compare the performance of ACMG/AMP-based variant interpretation software.
• To identify top-performing tools for variant prioritization in Mendelian disorders.
• To provide guidance for clinicians on selecting appropriate tools for variant analysis.

Main Methods:

• Benchmarked four ACMG/AMP tools (Franklin, InterVar, TAPES, Genebe) against LIRICAL using 151 expert-curated Mendelian disorder datasets.
• Evaluated tools based on top-N accuracy, retention rates, precision, recall, F1 scores, and AUC.
• Utilized bootstrap confidence intervals and Friedman tests for statistical validation.

Main Results:

• LIRICAL (68.21%) and Franklin (61.59%) demonstrated superior top-10 variant prioritization accuracy in Mendelian disorders.
• Tools with advanced phenotypic integration significantly outperformed those relying solely on genomic features.
• Statistical analysis confirmed the significant outperformance of LIRICAL and Franklin (p<0.0000).

Conclusions:

• Phenotype-driven variant interpretation tools, such as LIRICAL and Franklin, are more effective for Mendelian disorders.
• The study provides crucial data for clinicians selecting variant interpretation software.
• Open availability of data and code ensures reproducibility and facilitates further research.