Unlocking biological insight from single-cell data with an interpretable dual-stream foundation model

Deep learning foundation models are revolutionizing single-cell biology, yet learning holistic and discriminative representations from complex, high-dimensional data remains a central challenge. Although transformer-based single-cell language models have shown significant progress, they typically rely on a single input-encoding scheme, a practice that results in the loss of critical gene expression information and hinders the effective learning of global cellular representations. To address these challenges, we propose scDMC, an innovative single-cell pre-training framework.

At the input encoding level, scDMC employs a 'dual-stream' architecture by processing a rank-based sequence and an absolute expression-aware sequence in parallel to overcome the information loss inherent in single-encoding schemes. Building upon this, our framework synergistically optimizes local gene context and global cellular representation by combining masked language modeling at the gene level with momentum contrastive learning at the cell level. Pre-trained on only 2 million cells far fewer than the datasets used by mainstream models, scDMC sets a new state-of-the-art in multiple benchmark tasks, including cell annotation, clustering, and data integration. More importantly, we demonstrate that scDMC can uncover functional gene modules, infer cell-type-specific regulatory networks in a data-driven manner, and provides attention-guided biological insights.

scDMC provides a powerful and extensible foundation model framework for single-cell biology. By reconciling relative gene importance with absolute expression magnitude, it enhances representation fidelity. Its robust performance and biologically meaningful interpretations offer a new paradigm for data-driven discovery, facilitating deeper insights into cellular heterogeneity and gene regulatory mechanisms across diverse biological contexts.