Scaling Large Language Models for Next-Generation Single-Cell Analysis | JoVE Visualize

Area of Science:

Computational Biology
Genomics
Artificial Intelligence

Background:

Single-cell RNA sequencing (scRNA-seq) reveals cellular diversity but current models lack scalability and text integration.
Existing single-cell foundation models (scFMs) struggle with diverse tasks and combining transcriptomic data with biological text.
The Cell2Sentence (C2S) framework represents scRNA-seq profiles as text, offering a foundation for integrating diverse data.

Purpose of the Study:

To develop a scalable and versatile foundation model for single-cell analysis by integrating transcriptomic and textual data.
To enhance the capabilities of LLMs for biological research by training on a large corpus of biological data.
To demonstrate the utility of the C2S-Scale model in predicting biological responses and guiding experimental discovery.

Main Methods:

Trained a 27-billion-parameter LLM on over one billion tokens of transcriptomic data, biological text, and metadata using the C2S framework.
Employed reinforcement learning for targeted fine-tuning to improve performance on specific biological tasks.
Utilized a dual-context virtual screen to nominate drug candidates for context-selective biological effects.

Main Results:

C2S-Scale demonstrated consistent improvements in predictive and generative capabilities across various downstream tasks.
The model achieved strong performance in perturbation response prediction, natural language interpretation, and biological reasoning.
A virtual screen nominated silmitasertib (CX-4945) for context-selective antigen presentation upregulation, which was experimentally validated in human cell models.
C2S-Scale effectively integrates transcriptomic and textual data at an unprecedented scale, outperforming specialized scFMs and general LLMs.

Conclusions:

C2S-Scale represents a significant advancement in single-cell analysis, unifying diverse data types for next-generation research.
The model's ability to guide context-conditioned biological discovery opens new avenues for drug development and biological understanding.
C2S-Scale provides a powerful platform for creating "virtual cells" and pushing the boundaries of computational biology.