Calibration and evaluation of machine-learning algorithms for missense variant classification under ACMG/ClinGen...

Area of Science:

• Clinical Genetics
• Bioinformatics
• Genomic Variant Interpretation

Background:

• Missense variants frequently lead to variants of uncertain significance (VUS) in clinical genetics.
• The ClinGen framework now permits quantitative use of in silico predictors for variant classification (PP3/BP4).
• Newly developed machine-learning tools like AlphaMissense (AM) and PrimateAI-3D (PAI3D) require evaluation with real-world clinical data.

Purpose of the Study:

• To calibrate and assess the performance of AM and PAI3D within the ClinGen quantitative framework.
• To compare AM and PAI3D against the meta-predictor REVEL using real-world clinical datasets.
• To evaluate the impact of these tools on VUS reclassification.

Main Methods:

• Calibration of AM and PAI3D using the ClinGen quantitative framework.
• Performance assessment across ClinVar_2023, ClinGen, and Columbia Diagnostic Exome Sequencing (CDEX) datasets.
• Evaluation metrics included area under the curve (AUC), F1 scores, and VUS classification impact.

Main Results:

• REVEL exhibited the highest overall performance, especially for autosomal recessive and loss-of-function tolerant autosomal dominant variants.
• AM outperformed other tools for autosomal dominant variants within haploinsufficiency genes.
• Applying updated PP3/BP4 criteria reclassified over 10% of ClinGen VUS to likely pathogenic (LP) and <3% to likely benign (LB), with REVEL driving LP upgrades and AM driving LB downgrades.

Conclusions:

• AM, PAI3D, and REVEL demonstrate complementary strengths within the ClinGen framework.
• These tools support more accurate and mechanism-aware missense variant classification.
• Findings facilitate improved diagnostic yield and clinical decision-making for VUS.