A systematic analysis of regression models for protein engineering.

Area of Science:

• Protein engineering and computational biology.
• Application of machine learning (ML) in biological sciences.

Background:

• Protein optimization for industrial and pharmaceutical applications is a significant area of research.
• Machine learning is increasingly utilized to predict protein properties and guide experimental design.

Purpose of the Study:

• To evaluate the progress and challenges in applying machine learning for protein property prediction.
• To investigate the impact of different assessment criteria, regressors, and data representations on prediction performance.
• To address fundamental issues like sample bias and the importance of calibrated uncertainty in ML models for proteins.

Main Methods:

• Analysis of various regression metrics and generalization definitions for evaluating ML model performance.
• Identification and discussion of sample bias issues inherent in typical regression datasets.
• Emphasis on the necessity of calibrated uncertainty quantification in predictive models.

Main Results:

• Different assessment metrics and generalization definitions can lead to conflicting conclusions about ML model performance.
• Sample bias in datasets can create misleading impressions of regressor accuracy.
• The choice of regressor and data representation significantly influences prediction outcomes.

Conclusions:

• Accurate assessment of machine learning progress in protein optimization requires careful consideration of evaluation metrics and potential biases.
• Addressing sample bias and incorporating calibrated uncertainty are essential for reliable and interpretable protein property predictions.
• The findings underscore the need for robust evaluation frameworks in applying ML to protein engineering.