Genomic language model predicts protein co-regulation and function.

Area of Science:

• Genomics
• Machine Learning
• Bioinformatics

Background:

• Understanding gene-genomic context relationships is crucial for biological system engineering.
• Machine learning has advanced sequence-structure-function analysis but often overlooks genomic context.
• Evolutionary patterns in genomic contexts can reveal functional gene product relationships.

Purpose of the Study:

• To extend machine learning's capability to incorporate higher-order genomic context information.
• To develop a genomic language model (gLM) for learning latent functional and regulatory gene relationships.
• To encode biologically meaningful information from both protein sequences and their genomic surroundings.

Main Methods:

• Trained a genomic language model (gLM) on millions of metagenomic scaffolds.
• gLM learns contextualized protein embeddings integrating sequence and genomic context.
• Analyzed attention patterns to identify learned co-regulated functional modules (operons).

Main Results:

• gLM successfully encodes biologically relevant information, including enzymatic function and taxonomy.
• Attention analysis confirms gLM's ability to learn co-regulated gene modules like operons.
• Demonstrated unsupervised deep learning on metagenomic data effectively captures gene semantics and regulatory syntax.

Conclusions:

• gLM is a promising approach for encoding functional and regulatory gene information within genomic contexts.
• The model effectively uncovers complex relationships between genes in genomic regions.
• This deep learning strategy advances the understanding of gene interactions and biological system engineering.