Structure-informed protein language models are robust predictors for variant effects.

Area of Science:

• Computational Biology
• Protein Engineering
• Bioinformatics

Background:

• Protein language models (pLMs) learn evolutionary sequence distributions to predict variant effects.
• Sequence-only pLMs may not fully capture variant effects influenced by protein structure.
• There is a need to integrate structural context into pLMs for improved accuracy.

Purpose of the Study:

• To assess the impact of structural context on sequence-only pLMs for variant effect prediction.
• To develop a framework for incorporating protein structural information into pLMs.
• To enhance the prediction of protein fitness landscapes.

Main Methods:

• Introduced structure-informed pLMs (SI-pLMs) using cross-modality denoising for sequence and structure.
• Extended masked sequence denoising to jointly learn from sequence and structure data.
• Trained and evaluated SI-pLMs on deep mutagenesis scanning benchmarks.

Main Results:

• SI-pLMs achieved top performance on variant effect prediction benchmarks, outperforming competing methods.
• Structure-informed models were more effective than larger sequence-only models or those trained on more data.
• Case studies showed improved separability of fitness landscapes and conservation of key residues.

Conclusions:

• Integrating structural context into pLMs is more effective for capturing fitness landscapes than simply increasing model size or data.
• SI-pLMs offer a robust method for enhancing variant effect prediction by leveraging protein structure.
• The SI-pLM framework can be applied to existing pLMs without requiring structure data during prediction.