Do protein language models learn phylogeny?

Area of Science:

• Computational Biology
• Bioinformatics
• Machine Learning

Background:

• Deep machine learning models, particularly protein language models (pLMs), show promise in analyzing protein sequences.
• Classical phylogenetic tree inference relies on evolutionary relationships derived from sequence data.
• The integration of machine learning with traditional phylogenetics is an emerging area of research.

Purpose of the Study:

• To assess the ability of protein language models (pLMs) to discern phylogenetic relationships without explicit training.
• To compare the performance of pLMs (ESM2, ProtTrans, MSA-Transformer) against classical phylogenetic methods.
• To investigate the impact of sequence insertions and deletions (indels) on pLM performance in phylogenetic analysis.

Main Methods:

• Evaluation of ESM2, ProtTrans, and MSA-Transformer on 114 Pfam datasets.
• Comparison with established phylogenetic inference techniques.
• Analysis of performance across varying levels of sequence insertions and deletions (indels).

Main Results:

• The largest ESM2 model demonstrated superior performance in recovering phylogenetic relationships across diverse datasets and indel levels.
• pLMs generally agree with classical methods, with higher concordance observed in protein families exhibiting fewer indels.
• pLMs capture broader evolutionary relationships, with ESM2 showing particular strength in analyzing highly divergent sequences.

Conclusions:

• Protein language models, especially ESM2, can effectively infer evolutionary relationships and serve as a valuable complement to traditional phylogenetic methods.
• Sequence indels represent a key factor influencing the differences between pLM-based and classical phylogenetic approaches.
• A small subset of neurons within pLMs is sufficient to approximate phylogenetic distances, indicating efficient representation of evolutionary information.