GenSLMs: Genome-scale language models reveal SARS-CoV-2 evolutionary dynamics

Area of Science:

• Genomics
• Virology
• Artificial Intelligence

Background:

• Accurate and rapid identification of novel viral variants, such as SARS-CoV-2, is crucial for pandemic control.
• Existing methods for variant classification may not fully capture the complex evolutionary dynamics of rapidly mutating viruses.

Approach:

• Adaptation of large language models (LLMs) for genomic data to create genome-scale language models (GenSLMs).
• Pre-training GenSLMs on over 110 million prokaryotic gene sequences and fine-tuning on 1.5 million SARS-CoV-2 genomes.
• Demonstration of GenSLM scalability on advanced computing infrastructure, including GPU supercomputers and AI accelerators, achieving high computational performance.

Key Points:

• GenSLMs accurately and rapidly identify SARS-CoV-2 variants of concern.
• These models represent early examples of whole-genome foundation models applicable to diverse prediction tasks.
• The study showcases significant computational scaling, utilizing Zettaflops and PFLOPS for training.

Conclusions:

• GenSLMs offer a novel approach to understanding viral evolution and variant identification.
• The developed models provide initial scientific insights into tracking SARS-CoV-2 evolutionary dynamics.
• This work paves the way for applying large-scale biological data analysis to future pandemic preparedness.