Inferring context-specific site variation with evotuned protein language models

Multiple sequence alignments (MSAs) have been traditionally used for making inferences about site-specific diversity in proteins. Recent advancements in the field of artificial intelligence have highlighted the potential of protein language models (pLMs) to capture similar protein properties. Unlike MSAs, pLMs can make inferences from single sequences, without the need for a set of aligned sequences. In this study, we introduce a variation of the Context-Dependent Entropy metric, based on pLM embeddings instead of MSA input, to assess protein site conservation and variability. We test this metric using versions of two popular pLMs (ESM-2 and protT5) fine-tuned on the diversity of different Influenza A virus subtype hemagglutinin proteins. Our study demonstrates how our pLM entropy metric can capture which sites are more likely to change in a specific sequence context and how fine-tuning pLMs on a set of evolutionarily related proteins (evotuning) can improve the models' understanding of the group's diversity.