Understanding the heterogeneous performance of variant effect predictors across human protein-coding genes

Area of Science:

• Genomics and Bioinformatics
• Computational Biology
• Molecular Genetics

Background:

• Variant effect predictors (VEPs) are crucial computational tools for assessing genetic mutation impacts, particularly pathogenicity.
• Existing VEPs utilize diverse algorithms and training data, leading to potential variations in performance.
• Understanding VEP performance heterogeneity across different genes is essential for reliable variant interpretation.

Purpose of the Study:

• To evaluate the performance of 35 VEPs in distinguishing pathogenic from benign missense variants across 963 human genes.
• To investigate the gene-level heterogeneity in VEP performance and identify factors influencing it.
• To assess the predictability of gene-level VEP performance using machine learning models.

Main Methods:

• Assessed 35 VEPs on discriminating pathogenic and putatively benign missense variants across 963 human genes.
• Quantified gene-level performance using the area under the receiver operating characteristic curve (AUROC).
• Trained random forest models to predict gene-level AUROC based on gene-specific features.

Main Results:

• Observed significant gene-level heterogeneity in VEP performance (AUROC).
• Gene function, protein structure, and evolutionary conservation were identified as factors related to VEP performance.
• Intrinsic protein disorder significantly influenced apparent VEP performance, often inflating AUROC values for weakly conserved variants.

Conclusions:

• Gene-level features can help identify genes where VEP predictions may be more or less reliable.
• AUROC, while class-balance independent, has limitations for comparing VEP performance across diverse genes.
• The study highlights the need to consider gene-specific characteristics when interpreting VEP results.