Protein Set Transformer: A protein-based genome language model to power high diversity viromics

Area of Science:

• Genomics
• Bioinformatics
• Computational Biology

Background:

• Genomic data, especially from microbes and viruses, is rapidly increasing.
• Current homology-based functional analyses struggle with rapidly diverging viral and microbial genomes.
• Scalable and generalizable interpretation frameworks are needed.

Purpose of the Study:

• To develop a novel protein-based genome language model for interpreting viral genomes.
• To overcome limitations of homology-based methods in viral genomics.
• To establish a robust framework for viral genome analysis.

Main Methods:

• Developed Protein Set Transformer (PST), a language model that treats genomes as sets of proteins.
• Trained PST on a dataset of over 100,000 viral genomes.
• Evaluated PST's performance against homology-based and other language model approaches.

Main Results:

• PST outperformed existing methods in relating viral genomes based on shared protein content.
• PST demonstrated awareness of protein structure and function, clustering related proteins accurately.
• PST successfully clustered capsid proteins and viral late gene proteins.

Conclusions:

• PST is a valuable tool for viral genomics, ecology, and evolutionary studies.
• The PST framework shows potential as a foundation model for broader microbial genomics.
• This approach offers a scalable and generalizable method for interpreting genomic data.