Pool PaRTI: a PageRank-based pooling method for identifying critical residues and enhancing protein sequence...

Area of Science:

• Computational Biology
• Bioinformatics
• Machine Learning

Background:

• Protein language models (PLMs) generate variable-length residue embeddings.
• Downstream tasks require fixed-length protein vectors, necessitating pooling.
• Existing pooling methods often cause significant information loss.

Purpose of the Study:

• Develop a novel pooling method for more expressive protein embeddings.
• Enhance biological interpretability of protein representations.
• Improve performance in downstream machine learning tasks.

Main Methods:

• Introduce Pool PaRTI, a novel pooling method.
• Utilize internal transformer attention and PageRank for token importance weighting.
• Employ an unsupervised and parameter-free approach.

Main Results:

• Pool PaRTI prioritizes functionally critical residues.
• Achieved significant performance gains across four diverse protein ML tasks.
• Enhanced interpretability by identifying biologically relevant regions without structural data.
• Demonstrated robustness across different encoder-only PLMs.

Conclusions:

• Pool PaRTI offers a superior, interpretable pooling strategy for PLMs.
• The method improves predictive performance and biological insight.
• It is a generalizable and robust approach for protein representation learning.