Single-cell differential expression analysis between conditions within nested settings

Differential expression analysis provides insights into fundamental biological processes and with the advent of single-cell transcriptomics, gene expression can now be studied at the level of individual cells. Many analyses treat cells as samples and assume statistical independence. As cells are pseudoreplicates, this assumption does not hold, leading to reduced robustness, reproducibility, and an inflated type 1 error rate. In this study, we investigate various methods for differential expression analysis on single-cell data, conduct extensive benchmarking, and give recommendations for method choice. The tested methods include DESeq2, MAST, DREAM, scVI, the permutation test, distinct, and the t-test. We additionally adapt hierarchical bootstrapping to differential expression analysis on single-cell data and include it in our benchmark. We found that differential expression analysis methods designed specifically for single-cell data do not offer performance advantages over conventional pseudobulk methods such as DESeq2 when applied to individual datasets. In addition, they mostly require significantly longer run times. For atlas-level analysis, permutation-based methods excel in performance but show poor runtime, suggesting to use DREAM as a compromise between quality and runtime. Overall, our study offers the community a valuable benchmark of methods across diverse scenarios and offers guidelines on method selection.