Benchmarking large language models for cell typing in single-cell RNA-Seq.

Area of Science:

• Computational Biology
• Genomics
• Bioinformatics

Background:

• Single-cell RNA sequencing (scRNA-seq) generates high-dimensional data for cell type annotation.
• Current methods for cell type annotation using large language models (LLMs) lack a systematic framework and comprehensive performance evaluation.
• Traditional bioinformatics tools may struggle with the complexity and granularity of scRNA-seq data.

Purpose of the Study:

• To systematically benchmark LLMs against traditional tools for scRNA-seq cell type annotation.
• To identify optimal strategies for utilizing LLMs, including marker gene selection and ranking.
• To develop a robust and accessible solution for high-performance cell type annotation.

Main Methods:

• Benchmarking seven leading LLMs and three traditional bioinformatics tools.
• Utilizing 34 diverse human and mouse scRNA-seq datasets.
• Optimizing annotation accuracy through statistically significant marker genes ranked by log₂ fold change.

Main Results:

• LLMs demonstrated superior performance compared to traditional methods in cell type annotation.
• Top-performing LLMs included Kimi-k2, GPT-5, Claude-4.1, and Grok-4.
• An elite ensemble strategy achieved state-of-the-art accuracy, especially for fine-grained cell subtypes.

Conclusions:

• LLMs represent a significant advancement for cell type annotation in scRNA-seq research.
• The developed DeepCellSeek package provides a validated, high-performance, and user-friendly solution.
• This work establishes a roadmap for integrating LLMs into single-cell genomics for enhanced biological discovery.