Cell-o1 : training LLMs to solve single-cell reasoning puzzles with reinforcement learning

Area of Science:

• Computational Biology
• Genomics
• Artificial Intelligence

Background:

• Large language models (LLMs) show general reasoning but struggle with specialized tasks like single-cell RNA sequencing (scRNA-seq) data analysis.
• Cell type annotation is crucial for understanding cellular heterogeneity in scRNA-seq data.
• Current automated methods often lack batch-level context and explanatory reasoning.

Purpose of the Study:

• Introduce the CellPuzzles benchmark for batch-level cell type annotation.
• Develop a novel LLM, Cell-o1, to address limitations in current cell type annotation methods.
• Improve the accuracy and contextual understanding of cell type annotation in scRNA-seq data.

Main Methods:

• Reformulated cell type annotation as a batch-level reasoning task using the CellPuzzles benchmark.
• Developed Cell-o1, a 7B parameter LLM.
• Employed supervised fine-tuning with distilled reasoning traces and reinforcement learning with batch-level rewards for training Cell-o1.

Main Results:

• Off-the-shelf LLMs achieved low batch-level accuracy (19.0% for OpenAI o1).
• Cell-o1 significantly outperformed existing baselines, achieving over 73% improvement compared to OpenAI o1.
• Cell-o1 demonstrated strong generalization across diverse tissues, diseases, and donor conditions.

Conclusions:

• Cell-o1 represents a state-of-the-art approach for batch-aware cell type annotation in scRNA-seq data.
• The CellPuzzles benchmark facilitates the development and evaluation of LLMs for this task.
• Further analysis offers insights into LLM reasoning for biological data interpretation.