Self-supervised learning on millions of primary RNA sequences from 72 vertebrates improves sequence-based RNA...

Area of Science:

• Genomics
• Bioinformatics
• Computational Biology

Background:

• Language models excel in protein sequence analysis but are limited for genomic sequences, especially across species.
• Existing models struggle to leverage evolutionary information due to a lack of diverse genomic data.
• Understanding RNA splicing is crucial for gene regulation and requires advanced computational tools.

Purpose of the Study:

• To develop SpliceBERT, a novel language model for genomic sequences, focusing on RNA splicing.
• To leverage self-supervised learning (SSL) on diverse vertebrate RNA sequences for improved evolutionary information capture.
• To apply SpliceBERT to various downstream tasks, including variant effect prediction and splice site identification.

Main Methods:

• Pretraining SpliceBERT on primary RNA sequences from 72 vertebrates using masked language modeling.
• Utilizing learned hidden states and attention weights to characterize biological properties of splice sites.
• Evaluating SpliceBERT's performance on zero-shot variant effect prediction, human branchpoint prediction, and cross-species splice site prediction.

Main Results:

• Pretraining on diverse species enabled effective identification of evolutionarily conserved elements.
• SpliceBERT demonstrated proficiency in characterizing biological properties of splice sites.
• The model achieved success in zero-shot variant effect prediction, human branchpoint prediction, and cross-species splice site prediction.

Conclusions:

• Pretraining genomic language models on diverse species is crucial for capturing evolutionary insights.
• Self-supervised learning is a powerful approach for deciphering the regulatory logic of genomic sequences.
• SpliceBERT offers a promising tool for advancing RNA splicing research and understanding genomic regulation.