Transformer-Based Architectures for RNA-RNA Interaction Prediction

The accurate prediction of RNA-RNA interactions (RRIs) is a central challenge in computational RNA biology, owing to the complex, context-dependent, and long-range dependencies inherent in RNA sequences and structures. While recurrent neural networks and convolutional models have achieved notable success, they often struggle to efficiently capture global contextual relationships across long RNA sequences. Transformer-based architectures, originally developed for natural language processing, offer a powerful alternative through their self-attention mechanisms, enabling direct modeling of long-range dependencies without recurrence. This chapter provides a comprehensive overview of Transformer-based models for RRI prediction, including RNABERT and biomedical adaptations of BERT, discusses RNA-specific tokenization and pretraining strategies, and examines fine-tuning approaches for interaction prediction tasks. Case studies on benchmark datasets illustrate performance gains and biological interpretability, and current limitations and future research directions are outlined.