Survey of Protein Sequence Embedding Models

Area of Science:

• Computational biology
• Bioinformatics
• Protein science

Background:

• Natural language processing (NLP) algorithms have led to protein language models (PLMs) that encode protein sequences into fixed-size numerical vectors (embeddings).
• These PLMs are crucial for analyzing diverse protein lengths and amino acid compositions in various biological contexts.
• Representative models include Esm, Esm1b, ProtT5, SeqVec, and their derivatives like GoPredSim and PLAST.

Purpose of the Study:

• To survey and evaluate representative protein embedding models for diverse computational biology tasks.
• To assess the performance of these models in embedding the Saccharomyces cerevisiae proteome.
• To investigate the utility of PLMs for gene ontology (GO) annotation, human disease variant analysis, antimicrobial resistance correlation, and fungal mating factor analysis.

Main Methods:

• Surveyed and applied established protein embedding models (Esm, Esm1b, ProtT5, SeqVec) and their derivatives.
• Embedded the Saccharomyces cerevisiae proteome using these models.
• Utilized embeddings for GO annotation of uncharacterized proteins.
• Analyzed embeddings of human protein variants in relation to disease status.
• Correlated embeddings of Escherichia coli beta-lactamase TEM-1 mutants with experimental antimicrobial resistance data.
• Examined embeddings of fungal mating factors.

Main Results:

• All surveyed models indicated that uncharacterized yeast proteins are typically short (<200 amino acids), less rich in aspartate and glutamate, and enriched in cysteine.
• High-confidence GO term annotation was achieved for less than half of the uncharacterized yeast proteins.
• A statistically significant difference was observed in cosine similarity scores between benign and pathogenic human protein mutations.
• Embeddings of TEM-1 mutants showed low to no correlation with experimentally measured minimal inhibitory concentrations (MICs).

Conclusions:

• Protein language models offer valuable insights into protein characteristics and functions across different organisms and contexts.
• PLMs demonstrate potential for identifying patterns in uncharacterized proteins and predicting disease associations, though with limitations.
• The correlation between protein embeddings and quantitative functional measures like MICs requires further investigation and model refinement.