Poor Generalization by Current Deep Learning Models for Predicting Binding Affinities of Kinase Inhibitors

Area of Science:

• Computational chemistry
• Drug discovery
• Machine learning in pharmacology

Background:

• Neural networks are increasingly used to predict drug-receptor binding affinities, potentially accelerating drug discovery.
• Existing models using protein kinase sequences and SMILES strings report accurate quantitative inhibition predictions.
• Concerns remain regarding the generalizability of these neural network models to novel, unseen data.

Approach:

• A Convolutional Neural Network (CNN) model was developed, similar to previously reported architectures.
• The CNN was evaluated on four standard datasets for inhibitor/kinase binding predictions.
• Model performance was assessed under random data splitting versus inhibitor-grouped data splitting.

Key Points:

• The CNN achieved comparable performance to prior models when data points were randomly split between training and testing sets.
• Performance drastically decreased when all data for a specific inhibitor was grouped in either the training or testing set, indicating information leakage.
• The model demonstrated no significant generalization ability, performing no better than simple baseline models like tokenized SMILES or nearest neighbor predictions.

Conclusions:

• The observed lack of generalization suggests current CNN models do not truly learn molecular interactions for binding affinity prediction.
• Information leakage, particularly when data is not properly segregated, inflates performance metrics.
• Richer, structure-based molecular encodings are necessary for developing models capable of reliable prospective predictions of new drug candidates.