Benchmarking DNA Sequence Models for Causal Regulatory Variant Prediction in Human Genetics

Area of Science:

• Genomics
• Machine Learning
• Computational Biology

Background:

• Machine learning (ML) is crucial for identifying causal genetic variants in Mendelian and complex traits.
• Current ML approaches include supervised sequence-to-function and self-supervised DNA language models.
• A lack of curated datasets with accurate labels hinders benchmarking, especially for non-coding variants.

Purpose of the Study:

• Introduce TraitGym, a curated dataset of regulatory genetic variants for benchmarking ML models.
• Evaluate the performance of various ML models in predicting causal variants.
• Provide insights into the capabilities and limitations of different prediction strategies.

Main Methods:

• Developed TraitGym, a dataset of causal/candidate regulatory variants and controls for 113 Mendelian and 83 complex traits.
• Framed variant prediction as a binary classification task.
• Benchmarked supervised, self-supervised, hybrid, and ensemble ML models.

Main Results:

• Alignment-based models (CADD, GPN-MSA) performed well for Mendelian and complex disease traits.
• Functional genomics models (Enformer, Borzoi) excelled in complex non-disease traits.
• Evo2 showed scalability benefits but lagged behind alignment models, especially for enhancer variants.

Conclusions:

• TraitGym facilitates comprehensive benchmarking of ML models for causal variant identification.
• Different ML approaches show varying performance depending on trait type and variant class.
• The dataset and benchmark provide valuable resources for advancing genetic variant interpretation.