Protein language models are accidental taxonomists.

Area of Science:

• Bioinformatics
• Computational Biology
• Machine Learning in Biology

Background:

• Protein-protein interactions (PPIs) are crucial for biological functions.
• Experimental PPI characterization is resource-intensive.
• Computational methods, especially protein language models (pLMs), offer high-throughput PPI prediction but face challenges with multi-species data.

Purpose of the Study:

• To investigate why computational models show unexpectedly high performance on multi-species PPI datasets.
• To introduce and validate the "accidental taxonomist" hypothesis.
• To propose strategies for improving multi-species PPI prediction.

Main Methods:

• Analysis of standard multi-species PPI datasets to identify biases in positive and negative sample origins.
• Utilizing pLM embeddings to assess taxonomic relatedness of protein pairs.
• Implementing a strategic negative sampling strategy (same-species pairs) to mitigate taxonomic bias.
• Comparing model performance before and after strategic sampling.

Main Results:

• Standard multi-species PPI datasets exhibit a bias where positive pairs share taxonomic origins, unlike random negatives.
• pLM embeddings can accurately predict taxonomic origin, indicating models learn phylogeny instead of PPIs.
• Restricting negative examples to same-species pairs significantly reduces model performance, confirming the accidental taxonomist hypothesis.
• Strategically trained models still outperform single-species models, highlighting the potential of curated multi-species data.

Conclusions:

• The "accidental taxonomist" is a significant confounder in multi-species PPI prediction, where models exploit phylogenetic signals.
• Careful curation of multi-species datasets, particularly in negative sampling, is essential for reliable computational PPI prediction.
• The accidental taxonomist effect is likely relevant to other supervised learning tasks involving protein datasets across different species.