Scaling Large Language Models for Next-Generation Single-Cell Analysis

Area of Science:

• Computational Biology
• Genomics
• Artificial Intelligence

Background:

• Single-cell RNA sequencing (scRNA-seq) reveals cellular diversity but current models lack scalability and text integration.
• Existing single-cell foundation models (scFMs) struggle with diverse tasks and combining transcriptomic data with biological text.
• The Cell2Sentence (C2S) framework represents scRNA-seq profiles as text, offering a foundation for integrating diverse data.

Purpose of the Study:

• To develop a scalable and versatile foundation model for single-cell analysis by integrating transcriptomic and textual data.
• To enhance the capabilities of LLMs for biological research by training on a large corpus of biological data.
• To demonstrate the utility of the C2S-Scale model in predicting biological responses and guiding experimental discovery.

Main Methods:

• Trained a 27-billion-parameter LLM on over one billion tokens of transcriptomic data, biological text, and metadata using the C2S framework.
• Employed reinforcement learning for targeted fine-tuning to improve performance on specific biological tasks.
• Utilized a dual-context virtual screen to nominate drug candidates for context-selective biological effects.

Main Results:

• C2S-Scale demonstrated consistent improvements in predictive and generative capabilities across various downstream tasks.
• The model achieved strong performance in perturbation response prediction, natural language interpretation, and biological reasoning.
• A virtual screen nominated silmitasertib (CX-4945) for context-selective antigen presentation upregulation, which was experimentally validated in human cell models.
• C2S-Scale effectively integrates transcriptomic and textual data at an unprecedented scale, outperforming specialized scFMs and general LLMs.

Conclusions:

• C2S-Scale represents a significant advancement in single-cell analysis, unifying diverse data types for next-generation research.
• The model's ability to guide context-conditioned biological discovery opens new avenues for drug development and biological understanding.
• C2S-Scale provides a powerful platform for creating "virtual cells" and pushing the boundaries of computational biology.