CodonMoE: DNA Language Models for mRNA Analyses.

Area of Science:

• Genomics
• Computational Biology
• Machine Learning

Background:

• Genomic language models (gLMs) face efficiency challenges with DNA and RNA data.
• Current approaches involve separate models or large multi-modal architectures, both computationally intensive.

Purpose of the Study:

• To introduce CodonMoE, a lightweight adapter to enable DNA models to analyze RNA without specific pretraining.
• To address the computational burden of genomic language modeling.

Main Methods:

• Developed CodonMoE, an adaptive mixture of codon reformative experts.
• Theoretically analyzed CodonMoE as a universal codon-level approximator.
• Augmented existing DNA models with CodonMoE for RNA prediction tasks.

Main Results:

• DNA models with CodonMoE significantly outperformed unmodified DNA models on four RNA prediction tasks (stability, expression, regulation).
• The HyenaDNA+CodonMoE models achieved state-of-the-art results with 80% fewer parameters than specialized RNA models.
• Maintained sub-quadratic complexity while improving performance.

Conclusions:

• CodonMoE offers an efficient method to unify genomic language modeling by leveraging DNA models for RNA analysis.
• This approach reduces computational overhead and parameter requirements.
• Preserves modality-specific performance advantages, paving the way for more accessible genomic AI.