VenusMutHub: A systematic evaluation of protein mutation effect predictors on small-scale experimental data

Area of Science:

• Protein engineering
• Computational biology
• Biochemistry

Background:

• Computational models are vital for predicting mutation effects in protein engineering.
• Current evaluations often rely on deep mutational scanning (DMS) with surrogate readouts, which may not reflect real-world biochemical properties.
• High-throughput methods are unsuitable for many proteins and industrial applications, creating a need for small-scale, diverse datasets.

Purpose of the Study:

• To establish a comprehensive benchmark for evaluating computational models predicting mutation effects in proteins.
• To address the lack of suitable datasets for assessing model performance on small-scale experimental data with direct biochemical measurements.
• To provide practical guidance for selecting appropriate prediction methods in protein engineering.

Main Methods:

• Curated 905 small-scale experimental datasets from literature and public databases, covering 527 proteins.
• Included datasets with direct biochemical measurements (stability, activity, binding affinity, selectivity) instead of surrogate readouts.
• Evaluated 23 computational models employing sequence-based, structure-informed, and evolutionary approaches.

Main Results:

• VenusMutHub provides a rigorous assessment of computational model performance using direct biochemical data.
• The benchmark spans diverse proteins and functional properties, reflecting realistic industrial scenarios.
• The study facilitates a more accurate evaluation of models for predicting specific functional changes due to mutations.

Conclusions:

• VenusMutHub serves as a crucial resource for benchmarking protein engineering models.
• Direct biochemical measurements in small-scale datasets offer a more reliable evaluation framework.
• This benchmark aids in selecting effective computational tools for targeted protein design and engineering.