BulkFormer: A large-scale foundation model for bulk transcriptomes

Foundation models are transforming transcriptome analysis, yet most RNA sequencing (RNA-seq) foundation models are pretrained on sparse single-cell RNA-seq data, which typically captures only ∼3,000 genes per cell. This creates a need for models tailored to bulk transcriptomes, a modality that profiles ∼16,000 genes per sample and supports clinical and tissue-level analyses. Here, we present BulkFormer, a foundation model for bulk transcriptome analysis. BulkFormer contains ∼150 million parameters, covers 20,010 protein-coding genes, and is pretrained on 581,503 human bulk RNA-seq profiles. Its hybrid encoder combines a graph neural network to model explicit gene-gene relationships with a Performer module to capture global expression dependencies. Across five downstream tasks, BulkFormer outperformed existing single-cell foundation models while requiring substantially lower training cost. These results identify pretraining data modality as a key determinant of foundation model performance and establish BulkFormer as a framework for bulk transcriptome modeling. A record of this paper's transparent peer review process is included in the supplemental information.