Parameter-free representations outperform single-cell foundation models on downstream benchmarks

Area of Science:

• Computational Biology
• Genomics
• Bioinformatics

Background:

• Single-cell RNA sequencing (scRNA-seq) data possesses inherent statistical structure, driving the development of advanced foundation models.
• Transformer-based models like TranscriptFormer learn gene expression patterns by embedding genes into latent spaces, achieving state-of-the-art (SOTA) results in various biological tasks.

Purpose of the Study:

• To investigate if SOTA performance in analyzing scRNA-seq data can be achieved using computationally efficient, interpretable methods instead of complex deep learning representations.
• To evaluate the efficacy of simple normalization and linear modeling pipelines against established foundation models.

Main Methods:

• Development of interpretable pipelines utilizing careful data normalization techniques.
• Application of linear methods for gene expression data analysis.
• Benchmarking against SOTA foundation models on established datasets and out-of-distribution tasks.

Main Results:

• Simple linear pipelines achieved SOTA or near-SOTA performance across multiple benchmarks for scRNA-seq data analysis.
• These methods outperformed foundation models on out-of-distribution tasks, including novel cell types and organisms not present in training data.
• Demonstrated that linear representations can effectively capture the biology of cell identity.

Conclusions:

• Computationally intensive deep learning representations are not always necessary for achieving high performance in scRNA-seq data analysis.
• Rigorous benchmarking is crucial for evaluating the true capabilities of computational methods in genomics.
• Interpretable, linear models offer a powerful and efficient alternative for understanding cell identity from gene expression data.