Large-scale multi-omic biosequence transformers for modeling protein-nucleic acid interactions.

Area of Science:

• Bioinformatics
• Computational Biology
• Molecular Biology

Background:

• Transformer models have advanced bioinformatics, excelling in single-omic tasks like protein structure prediction.
• Current models trained on single data types (proteins or nucleic acids) struggle to capture cross-modal biological interactions.

Purpose of the Study:

• Introduce OmniBioTE, the largest open-source multi-omic foundation model for biological sequences.
• To demonstrate the capability of multi-omic training for capturing gene-protein relationships and interaction properties.

Main Methods:

• Trained OmniBioTE on over 250 billion tokens of mixed protein and nucleic acid sequence data.
• Evaluated OmniBioTE's performance on predicting binding free energy and identifying protein residues involved in interactions.

Main Results:

• OmniBioTE learns joint representations mapping genes to protein sequences from unlabeled data.
• Achieved state-of-the-art results in predicting binding free energy for protein-nucleic acid interactions.
• Demonstrated emergent learning of structural information and superior performance-per-FLOP compared to single-omic models.

Conclusions:

• Multi-omic transformer models offer a unified approach for biological sequence analysis.
• OmniBioTE serves as a powerful foundation model for advancing multi-omic biological discovery and understanding.