Resolving data bias improves generalization in binding affinity prediction

Area of Science:

• Computational chemistry
• Structural biology
• Machine learning

Background:

• Accurate prediction of protein-ligand binding affinities is crucial for computational drug design.
• Existing deep learning models often show inflated performance due to train-test data leakage from the PDBbind database and benchmark datasets.
• This leakage overestimates the generalization capabilities of current binding affinity prediction models.

Purpose of the Study:

• To address the issue of train-test data leakage in binding affinity prediction datasets.
• To develop a reliable benchmark dataset and a robust model for evaluating generalization capabilities.
• To identify the true performance of deep learning models in computational drug design.

Main Methods:

• Developed PDBbind CleanSplit, a curated training dataset using a novel structure-based filtering algorithm to eliminate data leakage and redundancy.
• Retrained existing top-performing deep learning models on the CleanSplit dataset.
• Developed a novel graph neural network model utilizing sparse graph modeling of protein-ligand interactions and transfer learning from language models.

Main Results:

• Retraining existing models on CleanSplit resulted in a substantial drop in performance, confirming the significant impact of data leakage.
• The proposed graph neural network model maintained high performance on the CleanSplit benchmark.
• The graph neural network model demonstrated strong generalization capabilities on strictly independent test datasets.

Conclusions:

• The performance of many current deep learning models for binding affinity prediction is largely overestimated due to data leakage.
• The PDBbind CleanSplit dataset provides a more realistic evaluation of model generalization.
• The developed graph neural network model offers a promising approach for accurate and generalizable binding affinity prediction in drug design.