Efficient generative modeling of protein sequences using simple autoregressive models.

Area of Science:

• Computational biology
• Bioinformatics
• Machine learning in protein science

Background:

• Generative models are key for protein design and extracting insights from vast protein sequence databases.
• Existing methods like Boltzmann machines and deep generative models are computationally intensive.

Purpose of the Study:

• To introduce simple autoregressive models as computationally efficient alternatives for protein sequence generation and analysis.
• To demonstrate their comparable accuracy and improved mathematical tractability.

Main Methods:

• Development and application of simple autoregressive generative models.
• Comparative analysis of computational cost against existing deep generative models.
• Utilizing model entropy for estimating functional sequence space size.

Main Results:

• Autoregressive models achieve accuracy similar to complex methods at 100-1000x lower computational cost.
• Models enable straightforward calculation of sequence probabilities and functional space size.
• Analysis of response regulators revealed ~10^68 possible sequences, a tiny fraction of total sequences.

Conclusions:

• Simple autoregressive models present a computationally efficient and mathematically advantageous approach to protein sequence generation and analysis.
• These models facilitate exploration of vast protein sequence spaces, highlighting both potential and challenges.
• The findings underscore the utility of generative models in understanding protein family diversity.