Biophysics-based protein language models for protein engineering

Area of Science:

• Computational biology
• Protein engineering
• Machine learning

Background:

• Protein language models (PLMs) trained on evolutionary data are effective for predicting protein sequence, structure, and function.
• Existing PLMs often neglect crucial biophysical factors that govern protein behavior.
• Integrating biophysical principles can enhance the predictive power of PLMs.

Purpose of the Study:

• To develop a novel protein language model framework, mutational effect transfer learning (METL), that incorporates biophysical modeling.
• To pretrain neural networks on biophysical simulation data to capture sequence-energetics relationships.
• To fine-tune METL on experimental data for improved prediction of protein properties.

Main Methods:

• Developed the METL framework, combining advanced machine learning with biophysical modeling.
• Pretrained transformer-based neural networks on biophysical simulation data.
• Fine-tuned the model on experimental sequence-function data for property prediction.

Main Results:

• METL effectively captures fundamental relationships between protein sequence, structure, and energetics.
• The model demonstrates superior performance in protein engineering tasks, including generalization from small datasets and position extrapolation.
• METL successfully designed functional green fluorescent protein variants using only 64 training examples.

Conclusions:

• METL represents a significant advancement in protein language models by integrating biophysical insights.
• This biophysics-based approach shows great potential for accelerating protein engineering and design.
• METL offers a powerful alternative to evolutionary-based models, particularly for tasks requiring deep biophysical understanding.