scGPT: toward building a foundation model for single-cell multi-omics using generative AI.

Area of Science:

• Computational Biology
• Genomics
• Single-cell Analysis

Background:

• Generative pretrained models show success in language and vision.
• Foundation models, built on large datasets and transformers, are a promising approach.
• Cellular biology, like language, can be analyzed by its fundamental units (genes).

Purpose of the Study:

• To explore the application of foundation models in advancing cellular biology and genetic research.
• To construct a foundation model for single-cell biology using generative pretrained transformers.
• To demonstrate the model's ability to distill biological insights and improve downstream tasks.

Main Methods:

• Utilized a generative pretrained transformer architecture.
• Trained the model, scGPT, on a large dataset of over 33 million single-cell sequencing profiles.
• Applied transfer learning for optimizing performance in various downstream applications.

Main Results:

• scGPT effectively distills critical biological insights regarding genes and cells.
• The model shows superior performance in diverse downstream applications after transfer learning.
• Demonstrated utility in cell type annotation, multi-batch and multi-omic integration, perturbation response prediction, and gene network inference.

Conclusions:

• Foundation models, exemplified by scGPT, hold significant potential for advancing single-cell biology and genetic research.
• scGPT provides a powerful tool for extracting biological knowledge from large-scale single-cell data.
• The model's adaptability through transfer learning enables high performance across a spectrum of complex biological analyses.