Evolutionary Reasoning Does Not Arise in Standard Usage of Protein Language Models.

Area of Science:

• Computational Biology
• Bioinformatics
• Machine Learning

Background:

• Deep learning models excel at individual protein sequence analysis but struggle with generalization.
• Current methods like masked language modeling focus on single sequences, limiting intersequence reasoning crucial for phylogenetics.

Purpose of the Study:

• To develop a novel deep learning architecture, Phyla, for explicit intersequence reasoning in biological foundation models.
• To enhance generalizability and performance in phylogenetic inference and other computational biology tasks.

Main Methods:

• Developed Phyla, a hybrid state-space transformer architecture operating on phylogenetic tree representations.
• Introduced a novel tree loss function to train the model for phylogenetic tree reconstruction.

Main Results:

• Achieved state-of-the-art performance on sequence reasoning benchmarks and phylogenetic tree reconstruction.
• Accurately reclassified archaeal organisms, like Lokiarchaeota, placing them closer to bacteria, aligning with recent phylogenetic findings.

Conclusions:

• Phyla advances molecular sequence reasoning by emphasizing structured reasoning over memorization.
• The architecture shows significant potential for improving protein sequence analysis and phylogenetic inference.