Information Leakage in Enzyme Substrate Prediction

Enzymes are essential catalysts in many cellular processes. Understanding their interactions with small molecules, such as regulators, cofactors, and most importantly, substrates, is crucial for understanding the biochemical processes that occur in cells. Correctly interpreting the roles of small molecules that interact with enzymes is key to elucidating enzyme function. Recently, enzyme-small molecule interaction prediction has attracted growing interest from computational methods, especially deep learning. As a result, researchers have published several datasets and numerous models with remarkable performance.

In this work, we critically examine one of the most popular datasets and four models trained on it, identifying similarity-induced information leakage that may overinflate reported model performance in out-of-distribution applications. We show that the inspected models are susceptible to information leakage, and their performance is not better than statistical baselines when the leakage is removed. Furthermore, controlling for leakage due to protein sequence similarity alone still yields high predictive performance, whereas performance decreases substantially when common substrates shared between training and test sets are removed from one of them, and approaches random prediction when structurally similar ligands are separated between splits. Thus, the investigated models generalize considerably better across enzyme sequence space than across ligand chemical space, indicating that their reported performance depends strongly on the small-molecule similarity between training and evaluation data.

All data splits we calculated in this study are available on Zenodo (DOI: 10.5281/zenodo.18786394). The code is available on GitHub and backed up on Zenodo (DOI: 10.5281/zenodo.18788609).

Supplementary information is available at Bioinformatics online.