GenomeOcean: An Efficient Genome Foundation Model Trained on Large-Scale Metagenomic Assemblies

Area of Science:

• Genomics
• Bioinformatics
• Computational Biology

Background:

• Existing genome foundation models face limitations in efficiency, tokenization, architecture, and bias towards reference genomes.
• This restricts their ability to represent low-abundance microbes in the rare biosphere.

Purpose of the Study:

• To develop an advanced genome foundation model, GenomeOcean, capable of overcoming current limitations.
• To improve the representation of rare microbial species and enhance generalizability in genomic analysis.
• To facilitate natural product discovery and synthetic biology applications.

Main Methods:

• Trained a 4-billion-parameter generative model, GenomeOcean, on over 600 Gbp of metagenomic data.
• Utilized large-scale co-assemblies of metagenomic samples for training.
• Implemented a byte-pair encoding (BPE) tokenization strategy and architectural optimizations for efficient genome sequence generation.

Main Results:

• Achieved up to 150x faster genome sequence generation with high biological fidelity.
• Demonstrated superior representation of microbial species and generation of evolutionarily constrained protein-coding genes.
• Successfully discovered novel biosynthetic gene clusters (BGCs) and performed zero-shot synthesis of complete BGCs.

Conclusions:

• GenomeOcean establishes a new benchmark for metagenomic research, natural product discovery, and synthetic biology.
• The model offers a robust foundation for advancing precision medicine and understanding complex biological systems.
• Its ability to represent rare microbes and generate novel BGCs opens new avenues for scientific exploration.