A base measure of precision for protein stability predictors: structural sensitivity

Area of Science:

• Biophysics
• Computational Biology
• Protein Engineering

Background:

• Predicting protein fold stability changes (ΔΔG) is crucial for protein engineering and disease variant screening.
• Existing methods often rely on 3D structural data, but the impact of structural input precision is unclear.
• Quantifying the structural sensitivity of these prediction methods is needed.

Purpose of the Study:

• To assess the structural sensitivity of widely-used protein stability prediction methods.
• To understand how variations in protein structure input affect prediction accuracy.
• To establish best practices for evaluating protein stability prediction methods.

Main Methods:

• Saturated computational mutagenesis was performed on 87 structures from 25 proteins.
• Six popular protein stability prediction tools were analyzed.
• Structural sensitivity was computed for each method.

Main Results:

• Structural sensitivity varied significantly across methods, falling into two groups: 0.6–0.8 kcal/mol for local environment-aware methods and ~0.1 kcal/mol for machine learning methods.
• Prediction precision correlated with accuracy on mutation-type-balanced datasets.
• Model architecture, rather than protein structural differences, was the primary driver of sensitivity variation.

Conclusions:

• Protein stability prediction methods exhibit substantial differences in structural sensitivity, largely due to their underlying models.
• A new standard is proposed: evaluate ΔΔG using three protein structures and report the standard deviation to indicate precision.
• Machine learning methods may not fully leverage structural information, suggesting folded structures alone have limited value without considering unfolded states.