Protein Set Transformer: a protein-based genome language model to power high-diversity viromics

Area of Science:

• Genomics
• Bioinformatics
• Computational Biology

Background:

• Genomic data, particularly microbial and viral, is rapidly increasing.
• Traditional homology-based functional analyses struggle with rapidly diverging viral genomes and proteins, limiting data utility.
• Scalable and generalizable interpretation frameworks are essential for handling this data explosion.

Purpose of the Study:

• To introduce Protein Set Transformer (PST), a novel protein-based genome language model.
• To develop a scalable and generalizable framework for interpreting viral genomic data.
• To overcome limitations of homology-based methods in viral genomics.

Main Methods:

• Developed PST, a language model that treats genomes as sets of proteins, disregarding functional labels.
• Trained PST on a dataset exceeding 100,000 viral genomes.
• Evaluated PST's performance against homology- and other language model-based approaches.

Main Results:

• PST demonstrated superior performance in relating viral genomes based on shared protein content compared to existing methods.
• PST exhibited awareness of protein structure and function, clustering capsid proteins and late viral gene proteins accurately.
• The model successfully related viral genomes without relying on sparse functional labels.

Conclusions:

• PST is a valuable tool for viral genomics, ecology, and evolutionary research.
• The PST framework offers a robust method for interpreting viral genomic data.
• PST has the potential to serve as a foundational model for microbial genomics.