Single-cell omics arena: evaluation of large language models for automatic cell-type annotations on single-cell omics...

Area of Science:

• Single-cell genomics
• Computational biology
• Artificial intelligence in biology

Background:

• Single-cell sequencing allows detailed tissue analysis but cell-type annotation is challenging.
• Automated methods are needed to overcome labor-intensive, expert-driven cell identification.
• Large language models (LLMs) show promise for extracting biological knowledge and automating annotations.

Purpose of the Study:

• To evaluate the efficacy of modern LLMs for automated cell-type annotation.
• To establish a comprehensive benchmark for assessing LLM performance on single-cell RNA sequencing (scRNA-seq) data.
• To develop advanced prompting techniques and cross-modality translation for improved annotation.

Main Methods:

• Compiled 11 scRNA-seq datasets and 1226 annotation tasks for benchmarking.
• Evaluated eight different LLMs on their cell-type annotation capabilities.
• Developed domain-specific chain-of-thought prompting and VAE-based cross-modality translation.

Main Results:

• Established the Single-cell Omics Arena benchmark for evaluating LLMs in cell-type annotation.
• Demonstrated LLMs' ability to perform automated cell-type annotation using interpretable features like gene names.
• Showcased enhanced accuracy via chain-of-thought prompting and extended applicability to non-RNA data using VAEs.

Conclusions:

• LLMs offer a powerful tool for automating cell-type annotation in scRNA-seq data.
• The Single-cell Omics Arena benchmark provides valuable insights into LLM performance.
• Cross-modality translation enables LLM application to diverse single-cell omics data types.