Analysing open-source protein folding models for nanobody binding prediction

Antibody-based therapeutics are a rapidly expanding class of treatments, with over 200 approved candidates and thousands in clinical trials. Computational pre-filtering using protein structure prediction models has the potential to reduce the cost of wet-lab screening, yet the relationship between model confidence measures and functional binding properties remains incompletely understood. Here, we evaluate whether confidence measures produced by contemporary open-source protein structure prediction models are suitable for in silico screening of nanobody-antigen interactions.

We benchmark Boltz-2, Chai-1, IntFold, and AlphaFold3 using two complementary tasks: (i) ranking true nanobody-antigen binding complexes above non-binding bait pairs across 17 antigens, and (ii) detecting out-of-distribution sequences generated by alanine substitution of all complementarity-determining region residues. We further assess confidence measure sensitivity through progressive alanine mutagenesis on 13 nanobody-antigen complexes spanning the range of CDR3 lengths in our dataset and evaluate generalizability using data from a camelid immunization campaign against CD33.

Boltz-2-derived confidence measures achieved the highest median performance for identifying true binders. Local confidence measures, including pLDDT and interface- or CDR-focused metrics, were most effective at detecting out-of-distribution sequences and exhibited the greatest sensitivity to mutations. No single confidence measure performed best across both tasks, and all evaluated protein structure prediction models showed limited generalization to previously unseen antigens.

Our results suggest that robust in silico nanobody candidate selection should combine complementary global and local confidence measures rather than relying on a single metric. These findings provide practical guidance for integrating open-source protein structure prediction models into AI-driven nanobody discovery pipelines while highlighting the need for improved generalization across antigens.