Generalizable deep-learning-based mRNA-protein interaction prediction strongly depends on protein diversity

Area of Science:

• Bioinformatics
• Computational Biology
• Molecular Biology

Background:

• RNA-binding proteins (RBPs) are crucial for gene expression regulation via mRNA-protein interactions (mRPIs).
• Deep learning models predict mRPIs using sequence data, but reported high accuracy may be inflated by data leakage.
• Protein structure, not just sequence, influences RNA recognition, posing challenges for sequence-only prediction models.

Purpose of the Study:

• To systematically investigate data leakage and generalization issues in sequence-based mRPI prediction models.
• To develop a rigorous evaluation framework and benchmark dataset for assessing mRPI prediction model performance.
• To determine if advanced encoding strategies improve generalization to unseen RBPs.

Main Methods:

• Constructed a benchmark dataset from CLIP experiments for mRPI prediction.
• Implemented random interaction-level and RBP-aware data splitting strategies.
• Evaluated attention-based deep learning models using one-hot, language model, and structure-aware RBP encodings.

Main Results:

• High performance was observed only when test RBPs were present in training data, indicating poor generalization to unseen RBPs.
• Performance significantly dropped when predicting interactions for proteins not included in the training set.
• Even with advanced encodings, models failed to generalize effectively, suggesting sequence alone is insufficient.

Conclusions:

• Existing sequence-based mRPI prediction models are largely overfitted and do not generalize well to novel RBPs.
• A rigorous RBP-aware evaluation framework revealed limitations in current prediction capabilities.
• Advancing mRPI prediction requires incorporating protein diversity and features beyond primary sequence information.