Adversarial Sequence Mutations in AlphaFold and ESMFold Reveal Nonphysical Structural Invariance, Confidence...

Area of Science:

• Structural biology
• Computational biology
• Protein structure prediction

Background:

• AlphaFold revolutionized protein structure prediction, enabling tools for protein design and drug discovery.
• The generalizability of AlphaFold's learned principles versus template-based pattern matching is critical for its application scope.

Purpose of the Study:

• To systematically evaluate AlphaFold 3's robustness to mutations and deletions.
• To assess the reliability of AlphaFold 3's confidence metrics.
• To compare AlphaFold 3's performance with ESMFold under mutational stress.

Main Methods:

• Adversarial evaluation using point and deletion mutations across 200 proteins.
• Analysis of structural invariance to mutations, including destabilizing substitutions.
• Assessment of confidence metric correlation with structural quality and template similarity.

Main Results:

• AlphaFold 3 structures showed invariance to up to 40% residue mutations and 10% deletions.
• This invariance persisted even in fold-switching proteins, contrary to expectations.
• Confidence metrics were unreliable, correlating with training-set template quality.
• ESMFold demonstrated greater, though imperfect, mutational sensitivity.

Conclusions:

• AlphaFold 3 may heavily rely on memorized templates, potentially limiting its biophysical reasoning capabilities.
• Unreliable confidence metrics and template dependence have implications for interpreting mutation effects and model selection.
• Findings suggest caution in applying AlphaFold 3 to tasks requiring accurate prediction of mutation impacts.