Pre-training genomic language model with variants for better modeling functional genomics.

Area of Science:

• Genomics
• Computational Biology
• Functional Genomics

Background:

• Genomic language models (GLMs) learn from DNA sequences to create representations.
• Sequence-to-function (S2F) models predict gene expression from sequence data.
• Bridging GLMs and S2F models is crucial for individualized gene expression prediction.

Purpose of the Study:

• Introduce UKBioBERT, a GLM pre-trained on UK BioBank genetic variants.
• Enhance gene expression prediction by integrating UKBioBERT with S2F models.
• Investigate the relationship between genetic variants and gene expression variations.

Main Methods:

• Pre-trained UKBioBERT on large-scale DNA sequences from UK BioBank.
• Developed UKBioFormer and UKBioZoi by combining UKBioBERT with Enformer and Borzoi.
• Evaluated model performance on gene expression prediction and eQTL identification.

Main Results:

• UKBioBERT embeddings identify gene functions and improve cell line gene expression prediction.
• UKBioFormer and UKBioZoi show enhanced performance in predicting predictable gene expression levels.
• UKBioFormer effectively links genetic variants to expression variations, aiding in silico mutation analysis and eQTL identification.

Conclusions:

• Integrating GLMs and S2F models advances functional genomics research.
• UKBioBERT provides valuable embeddings for understanding gene expression predictability.
• The developed models offer improved generalization across cohorts and facilitate genetic variant-trait association studies.