Simulating 500 million years of evolution with a language model.

Area of Science:

• Biotechnology
• Computational Biology
• Protein Engineering

Background:

• Biological systems and protein functions are encoded through billions of years of evolution.
• Natural proteins represent a vast, yet incompletely understood, biological sequence space.

Purpose of the Study:

• To demonstrate that large-scale language models trained on evolutionary data can generate novel, functional proteins.
• To introduce ESM3, a multimodal generative language model for protein sequence, structure, and function.
• To explore the capabilities of ESM3 in generating proteins with desired characteristics, such as fluorescence.

Main Methods:

• Training a multimodal generative language model (ESM3) on large-scale evolutionary protein data.
• Utilizing complex prompts that combine sequence, structure, and function modalities for protein generation.
• Synthesizing and characterizing generated proteins, including fluorescent proteins.
• Assessing sequence identity to known proteins to quantify evolutionary distance.

Main Results:

• ESM3 successfully generated functional proteins distant from known protein families.
• A novel bright fluorescent protein was generated with 58% sequence identity to known fluorescent proteins.
• The evolutionary distance of the generated protein is estimated to be equivalent to 500 million years.

Conclusions:

• Large language models are powerful tools for exploring and expanding the protein sequence space.
• ESM3 can generate functional proteins that represent significant evolutionary novelty.
• This approach has the potential to accelerate protein engineering and the discovery of new biological functions.