Protein language models trained on multiple sequence alignments learn phylogenetic relationships.

Area of Science:

• Computational Biology
• Bioinformatics
• Machine Learning

Background:

• Self-supervised neural language models with attention are advancing biological sequence analysis.
• Protein language models like MSA Transformer utilize multiple sequence alignments (MSAs) for prediction tasks.
• Previous work showed success in contact prediction using row attentions in MSA Transformer.

Purpose of the Study:

• To investigate whether column attentions in MSA Transformer encode phylogenetic relationships.
• To determine if MSA-based language models can differentiate coevolutionary signals from phylogenetic noise.
• To assess the robustness of unsupervised contact prediction against phylogenetic noise.

Main Methods:

• Analyzing column attentions of MSA Transformer on biological sequence data.
• Generating synthetic MSAs with and without phylogeny using Potts models.
• Evaluating unsupervised contact prediction performance using MSA Transformer and Potts models.

Main Results:

• Combinations of MSA Transformer's column attentions strongly correlate with Hamming distances, indicating encoding of phylogenetic relationships.
• MSA-based language models successfully separate functional/structural coevolutionary signals from phylogenetic correlations.
• Unsupervised contact prediction using MSA Transformer demonstrated greater resilience to phylogenetic noise compared to Potts models.

Conclusions:

• MSA-based language models inherently encode detailed phylogenetic information.
• These models offer a powerful approach to disentangle evolutionary signals from historical contingency in biological sequences.
• The findings highlight the potential of protein language models for robust biological structure and function prediction.