Cracking the black box of deep sequence-based protein-protein interaction prediction

Area of Science:

• Computational Biology
• Bioinformatics
• Machine Learning

Background:

• Protein-protein interactions (PPIs) are fundamental to biological processes.
• Computational methods are developed to predict PPIs as cost-effective alternatives to experiments.
• Existing prediction methods report high accuracy, but reproducibility is often questionable.

Purpose of the Study:

• To systematically evaluate the reproducibility of deep learning models for PPI prediction.
• To investigate the impact of data leakage, sequence similarity, and node degree on model performance.
• To compare deep learning models against baseline machine learning approaches.

Main Methods:

• Deep learning models and basic machine learning models were employed for PPI prediction.
• The study involved systematic investigation of data leakage through set overlaps.
• Sequence similarity and node degree information were analyzed as key features.
• Performance was evaluated under conditions with and without data leakage.

Main Results:

• Random splitting of training and test sets leads to significant overestimation of model performance due to data leakage.
• When data leakage is prevented by minimizing sequence similarity, deep learning model performance becomes comparable to random chance.
• Baseline models utilizing sequence similarity and network topology achieve good performance with lower computational cost.
• Deep learning models primarily learn from sequence similarities and node degrees when data leakage is present.

Conclusions:

• Overestimated performance in PPI prediction is a common issue, often stemming from data leakage.
• Current deep learning approaches may not offer substantial improvements over simpler baseline methods when data leakage is controlled.
• Predicting PPIs for proteins with low sequence similarity to known proteins remains a significant challenge.
• Further experimental investigation of the 'dark' interactome and development of robust computational methods are essential.