DeOPUS: cellular deconvolution via optimized power-transformed unmixing with shrinkage

Single-cell RNA sequencing provides high-resolution insights into cellular heterogeneity but its widespread application is often constrained by high costs and technical complexity. Cellular deconvolution serves as a cost-effective alternative by computationally estimating cell-type proportions from bulk RNA-seq data. However, the extreme dynamic range and inherent heteroscedasticity of transcriptomic data pose significant challenges for accurate estimation. Here, we present deconvolution via optimized power-transformed unmixing with shrinkage (DeOPUS), a reference-based deconvolution method that introduces a hierarchical shrinkage transformation (HST) to robustly estimate cellular compositions. DeOPUS integrates multi-level adaptive priors, variance-stabilizing power transformations, and rank-based quantile normalization to mitigate the influence of outliers and high-variance technical noise. We systematically benchmark DeOPUS against eight state-of-the-art methods across 122 tissues and 12 organ systems. DeOPUS consistently outperforms all eight competitors by having mean Pearson ($r = 0.82$) and Spearman ($\rho = 0.77$) correlations. DeOPUS significantly surpasses the second-best method ($r = 0.75, \rho = 0.68$) while maintaining the lowest mean squared error (MSE = 0.007). Notably, DeOPUS ranks first in the vast majority of tissues and organ systems using all three metrics, demonstrating unrivaled robustness to increasing cellular complexity. More in-depth validation on 18 bulk datasets with experimentally determined cell-type proportions further confirms DeOPUS's strong performance. DeOPUS is the sole method to achieve positive correlations across all datasets, and achieves the highest accuracy for dominant cell-type identification. DeOPUS is available as an open-source R package at https://github.com/tinnlab/DeOPUS.