A significance score for protein-protein interaction models through random docking

Comparing accuracies of structural protein-protein interaction (PPI) models for different complexes on an absolute scale is a challenge, requiring normalization of scores across structures of different sizes and shapes. To help address this challenge, we have developed a statistical significance metric for docking models, called random-docking (RD) p-value. This score evaluates a PPI model based on how likely a random docking process is to produce a model of better or equal accuracy. The binding partners are randomly docked against each other a large number of times, and the probability of sampling a model of equal or greater accuracy from this reference distribution is the RD p-value. Using a subset of top predicted models from CAPRI (Critical Assessment of PRediction of Interactions) rounds over 2017-2020, we find that the ease of achieving a given root mean squared deviation or DOCKQ score varies considerably by target; achieving the same relative metric can be thousands of times easier for one complex compared to another. In contrast, RD p-values inherently normalize scores for models of different complexes, making them globally comparable. Furthermore, one can calculate RD p-values after generating a reference distribution that accounts for prior information about the interface geometry, such as residues involved in binding, by giving the random-docking process access the same information. Thus, one can decouple improvements in prediction accuracy that arise solely from basic modeling constraints from those due to the rest of the method. We provide efficient code for computing RD p-values at https://github.com/Grigoryanlab/RDP.