Impact of Single-Cell RNA Reference Selection for the Deconvolution of Breast Cancer Spatial Transcriptomics Datasets

Spot-based spatial transcriptomics (ST) allows for unbiased gene expression analysis within tissue architecture, overcoming the limitations of single-cell RNA sequencing (scRNA-seq) by preserving spatial context. However, the high spatial resolution in ST leads to cellular heterogeneity within spots, requiring computational deconvolution to infer cellular compositions. While scRNA-seq serves as a key reference for deconvolution, the impact of reference composition on its accuracy is still unclear. In this study, we systematically evaluate the impact of reference selection for cellular deconvolution and provide helpful guidelines for researchers. Pseudospots mimicking 55 μm Visium spots were generated from spatial transcriptomics data to evaluate global and cell type-specific deconvolution in primary (Xenium) and metastatic (MERFISH) breast cancer samples. Focusing on state-of-the-art deconvolution tools Cell2location and RCTD, we assess the influence of varying reference sizes, cell type distributions, and reference-ST pairings, as well as the usage of large breast cancer and cross-cancer atlases. Our findings demonstrate that even small references can yield accurate deconvolution, with RCTD and Cell2location exhibiting similar results. Spatial domains of prominent cell types like cancer and stromal cells were detected, although their contributions were systematically under- or over-estimated. Additionally, Reference-ST sample matching enhances accuracy compared to the usage of cross-patient references, while large diverse breast cancer atlases also provided reliable results. This study concluded that reference selection had modest effects on RCTD and Cell2location, but matching samples and large atlases stabilize outcomes. Nonetheless, performance varies between cell types and annotation levels, and deconvolution results should always be interpreted with caution.