The Impact of Experimental and Calculated Error on the Performance of Affinity Predictions

The accurate prediction of binding affinity between protein and small molecules with free energy methods, particularly the difference in binding affinities via relative binding free energy calculations, has undergone a dramatic increase in use and impact over recent years. The improvements in methodology, hardware, and implementation can deliver results with less than 1 kcal/mol mean unsigned error between calculation and experiment. This is a remarkable achievement and beckons some reflection on the significance of calculation approaching the accuracy of experiment. In this article, we describe a statistical analysis of the implications of variance (standard deviation) of both experimental and calculated binding affinities with respect to the unknown true binding affinity. We reveal that plausible ratios of standard deviation in experiment and calculation can lead to unexpected outcomes for assessing the performance of predictions. The work extends beyond the case of binding free energies to other affinity or property prediction methods.