Deep learning models for RNA secondary structure prediction (probably) do not generalize across families

Area of Science:

• Computational biology
• Bioinformatics
• Machine learning

Background:

• RNA secondary structure is crucial for its function.
• Machine learning models have shown promise in predicting RNA secondary structures.
• Existing models often excel at intra-family predictions but struggle with inter-family generalization.

Purpose of the Study:

• To evaluate the generalization capabilities of machine learning models for RNA secondary structure prediction.
• To introduce a rigorous inter-family cross-validation method.
• To demonstrate the limitations of current intra-family performance metrics.

Main Methods:

• Utilized convolutional neural networks (CNNs) to generate pseudo-free energy changes.
• Modeled predictions on structure mapping data.
• Developed and applied a novel inter-family cross-validation approach.

Main Results:

• CNNs effectively improve intra-family RNA secondary structure prediction accuracy.
• Intra-family performance is not indicative of true generalization ability.
• Many current machine learning models exhibit poor inter-family prediction performance.

Conclusions:

• Rigorous inter-family cross-validation is essential for assessing true model generalization.
• Existing machine learning models for RNA secondary structure prediction often fail to generalize across different RNA families.
• Further development is needed to create models that robustly predict RNA secondary structures across diverse families.