Interpretable pairwise distillations for generative protein sequence models.

Area of Science:

• Computational Biology
• Bioinformatics
• Machine Learning

Background:

• Generative models for protein sequences are crucial for applications like protein design and predicting mutational effects.
• Neural network (NN) architectures excel due to their ability to capture complex, higher-order interactions in protein data.

Purpose of the Study:

• To analyze two different NN models for protein sequences.
• To assess the proximity of these NN models to simpler pairwise distributions.
• To develop a method for extracting pairwise models from complex NN models.

Main Methods:

• Analysis of two distinct neural network (NN) architectures for protein sequence modeling.
• Application of an energy-based modeling framework to extract pairwise models from NN models.
• Evaluation of extracted pairwise models against original NN models on tasks such as mutational effect prediction.

Main Results:

• Extracted pairwise models successfully replicated the energies of the original NN models.
• Pairwise models demonstrated comparable performance to NN models in predicting mutational effects.
• Simpler, factorized models also achieved performance close to the original complex NN models.

Conclusions:

• Complex neural network models for protein sequences can be effectively approximated by simpler pairwise models.
• The energy-based modeling framework provides a viable method for extracting interpretable pairwise models from deep learning architectures.
• Simpler models offer a computationally efficient alternative without significant loss of predictive power for certain protein-related tasks.