Prediction of single cell expression from low-plex immunofluorescence images for guiding precision oncology

Spatial omics allows for comprehensive investigation of the tumor immune microenvironment (TIME). Stratifying patients by their TIMEs contributes with insights in tumor immune response and has the potential to guide treatment decision-making in the immuno-oncology setting. However, high-plex spatial omics approaches still suffer from high costs and limited direct clinical applicability. We address this issue by presenting a deep learning model, Image2Count, for deconvoluting molecular expression from low-plex immunofluorescence imaging. Explicitly, our model learns visual representations of cells in a contrastive manner, utilizing Graph Neural Networks to predict expressions from cell graphs, enabling the trained model to predict high-plex single-cell expression from just four marker images. We measure model performance using an ovarian cancer GeoMx dataset with "bulk" Region of Interest 72-plex protein counts, a Cyclic IF (t-CyCIF) 25-plex single-cell resolution colorectal cancer dataset, and a 960-plex RNA single-cell resolution CosMx non-small cell lung cancer dataset. Image2Count is able to predict distinct spatial expression patterns of subsets of tumor, immune and stromal cells, and displays a generally improved accuracy when considering neighborhoods of cells over single cells. Concordance of pathways enriched in true and predicted data indicates the ability to capture biologically relevant information. Our model paves the way for clinically implementable TIME stratification based on low-plex immunofluorescence images, and allows for standard single-cell analysis workflows to interpret multicellular expression data from regions of interest.