Evaluating sequence-to-function deep learning models for ancestry-stratified regulatory variant effect prediction...

Area of Science:

• Genomics
• Bioinformatics
• Computational Biology

Background:

• Sequence-to-function (S2F) deep learning models are vital for prioritizing non-coding regulatory variants.
• Existing models and data are predominantly European-centered, limiting their applicability to diverse populations.
• Multi-ancestry benchmarks are crucial to assess S2F score consistency across varying allele frequencies and linkage disequilibrium (LD) patterns.

Purpose of the Study:

• To evaluate the performance of S2F models (Borzoi, AlphaGenome) across ancestrally diverse populations.
• To determine if S2F scores consistently capture regulatory effects across African American (AA), Caribbean Hispanic (CH), and Non-Hispanic White (NHW) participants.
• To assess the models' ability to distinguish regulatory variants using fine-mapped data.

Main Methods:

• Evaluated Borzoi and AlphaGenome using eQTL data from the MAGENTA cohort (AA, CH, NHW).
• Benchmarked predictions against nominal eQTLs and ancestry-stratified fine-mapped variants.
• Utilized Spearman correlation, direction concordance, inter-model convergence, and distance-matched AUROC for evaluation.
• Conducted sensitivity analyses for minor allele frequency (MAF) and compared functional annotation overlap.

Main Results:

• Both models exhibited weak agreement with nominal eQTL effect sizes across ancestries.
• Agreement and discrimination improved significantly for high-confidence fine-mapped variants.
• The African American (AA) high-PIP variant set showed the highest discrimination (AUROC) for both models.
• Functional annotation revealed higher overlap with chromatin accessibility/contact data for AA high-PIP variants.

Conclusions:

• S2F models show limited predictive power for nominal eQTL effect sizes but effectively prioritize fine-mapped regulatory variants.
• The superior discrimination in the AA cohort suggests S2F models can be valuable for variant prioritization, particularly for promoter-proximal variants.
• Model performance is influenced by fine-mapping resolution, LD patterns, comparison variant selection, and annotation composition across ancestries.