scELMo: Embeddings from Language Models are Good Learners for Single-cell Data Analysis.

Area of Science:

• Computational Biology
• Bioinformatics
• Genomics

Background:

• Foundation Models (FMs) are increasingly used for single-cell data analysis, but with varying success.
• Existing methods often require extensive resources and task-specific training.

Purpose of the Study:

• To propose scELMo (Single-cell Embedding from Language Models), a novel method for single-cell data analysis.
• To leverage Large Language Models (LLMs) for generating metadata descriptions and embeddings.
• To enable zero-shot and fine-tuning capabilities for diverse single-cell tasks.

Main Methods:

• scELMo utilizes LLMs to generate embeddings from metadata descriptions.
• Combines LLM embeddings with raw single-cell data under a zero-shot learning framework.
• Employs a fine-tuning framework for advanced tasks like in-silico treatment analysis.

Main Results:

• scELMo performs cell clustering, batch effect correction, and cell-type annotation without new model training.
• Achieves superior performance compared to established FMs like scGPT and Geneformer.
• Demonstrates effectiveness in complex tasks such as perturbation modeling.

Conclusions:

• scELMo offers a computationally efficient and resource-light approach to single-cell data analysis.
• Represents a promising direction for developing domain-specific FMs for biological data.
• Outperforms existing LLM-based pipelines and large-scale FMs in evaluations.