A foundation model for nucleotide sequences.

Area of Science:

• Genomics
• Transcriptomics
• Bioinformatics
• Machine Learning

Background:

• Foundation models show promise in various fields, but their application in genomics and transcriptomics, especially integrating nucleotide sequences with annotations, is limited.
• Existing methods often focus on specific tasks like functional genomic element recognition, potentially hindering generalizability across species and biological contexts.

Purpose of the Study:

• Introduce OmniNA (Omni-applicable foundation model for nucleic acid), a self-supervised generative foundation model designed for genomics and transcriptomics.
• To jointly learn from nucleotide sequences and their annotations to improve semantic understanding and representation learning.
• To enhance model generalizability and performance in diverse biological tasks.

Main Methods:

• Trained OmniNA on 91.7 million nucleotide sequences and 197 million words of annotations, totaling 1076.2 billion bases across diverse species.
• Employed a self-supervised generative approach that leverages the complementarity of sequence and annotation data.
• Fine-tuned OmniNA using natural language paradigms for various downstream tasks.

Main Results:

• OmniNA effectively captures sequence grammar and annotation semantics, demonstrating robust transferability across nucleotide-level tasks.
• Achieved state-of-the-art or competitive performance in 23 benchmarks, including sequence detection and species classification.
• Learned representations aid in understanding mutation effects on DNA and RNA processing.

Conclusions:

• OmniNA represents a significant advancement in foundational modeling for genomics and transcriptomics by integrating annotation-aware learning.
• The model offers a powerful, publicly available resource for the research community.
• Annotation-aware learning enhances model generalizability and performance in biological sequence analysis.