MambaCell: A Self-Supervised Mamba Framework for Multi-Task Cell Representation Learning

Single-cell RNA sequencing (scRNA-seq) has become a groundbreaking tool in life science research. Moreover, advances in Large Language Models (LLMs) have greatly catalyzed the development of cellular foundation models in transcriptomics. However, existing scRNA-seq models built on LLMs predominantly employ transformer architectures, which are constrained by quadratic computational complexity during inference, limiting their ability to process long sequences. Furthermore, current cellular foundation models often focus on a single objective like Masked Language Modeling (MLM), which limits their ability to capture semantically rich and biologically meaningful representations. Critically, the field lacks an efficient and general framework for scalable cell representation learning. To address these limitations, we introduce MambaCell, a multi-task self-supervised learning framework based on the bidirectional Mamba architecture. This novel framework integrates two complementary self-supervised tasks including Masked Gene Modeling (MGM) and Contrastive Learning (CL), enabling it to learn robust cell representations from large-scale, un-labeled datasets with reduced inference costs. Experimental results demonstrate that MambaCell achieves superior or comparable performance to state-of-the-art (SOTA) models on a series of downstream tasks, including cell type annotation, disease-related cell classification, single-cell batch integration, etc. Notably, MambaCell achieves faster inference speed and higher memory efficiency compared to transformer-based models with comparable parameters. These superior performance and efficiency gains underscore the effectiveness of MambaCell's architectural innovation, establishing it as a scalable solution for large-scale single-cell transcriptomic analysis.