Are genomic language models all you need? Exploring genomic language models on protein downstream tasks

Area of Science:

• Computational biology
• Bioinformatics
• Machine learning in genomics and proteomics

Background:

• Large language models (LLMs) excel in genomic and proteomic tasks.
• Genomic language models (gLMs) can potentially predict protein functions from DNA sequences (CDS).
• The performance of gLMs on protein-related tasks is largely unexplored due to limited paired data.

Purpose of the Study:

• To evaluate the performance of gLMs on protein tasks compared to proteomic language models (pLMs).
• To investigate the benefits of joint genomic-proteomic models and different genomic tokenization strategies.
• To assess the potential of gLMs for a unified approach to genomics and proteomics.

Main Methods:

• Curated five datasets pairing proteins with their coding DNA sequences (CDS).
• Evaluated gLMs and pLMs on these datasets, comparing retrieved CDS versus sampling strategies.
• Trained and interpreted joint genomic-proteomic models and a new Nucleotide Transformer with 3mer tokenization.

Main Results:

• gLMs demonstrated competitive and, in some cases, superior performance compared to pLMs.
• Using retrieved CDS yielded better performance than sampling strategies.
• Joint models captured complementary sequence representations, outperforming individual models.
• A 3mer tokenized Nucleotide Transformer model improved protein task performance while maintaining genomic task performance.

Conclusions:

• gLMs are effective for protein sequence analysis, offering a powerful complement to pLMs.
• Joint genomic-proteomic models provide synergistic benefits by integrating diverse sequence information.
• Optimized tokenization (3mer) enhances gLM performance on protein tasks, supporting a unified biological sequence modeling approach.