Evaluating Machine Learning Affinity Prediction Methods on Novel Lead Optimization Data: A 15-PGDH Case Study

Recent machine learning (ML) methods for protein-ligand affinity prediction, such as Boltz-2 and AEV-PLIG, have been reported to approach free energy perturbation (FEP) accuracy on well-known benchmarks. Even short of that accuracy, ML-based potency models that fall between FEP and common docking scoring functions in accuracy and speed offer clear potential value. Here, we report a case study using recently disclosed data from four related ligand series targeting 15-hydroxyprostaglandin dehydrogenase (15-PGDH). We evaluate each ML method's predictive accuracy with a focus on when it could be used to influence synthetic priorities. AEV-PLIG performed poorly across most series, and adding literature 15-PGDH inhibitors to its training did not improve performance. Boltz-2 performed well on series most similar to published inhibitors in its training, achieving accurate cofolding (median ligand RMSD ≤0.81 Å) and rank order. However, for a series exploring a new region of the binding pocket, Boltz-2 sometimes generated less accurate poses and predicted all ligands in the series to be less potent than a published exemplar, despite experimental potency gains of up to 500x. These results highlight both the promise in well-characterized chemical space and limitations in the novel regions where computational guidance is most needed.