A deep learning-based multiscale integration of spatial omics with tumor morphology

Spatial Transcriptomics (spTx) offers unprecedented insights into the spatial arrangement of the tumor microenvironment, tumor initiation/progression and identification of new therapeutic target candidates. However, spTx remains unlikely to be routinely used in the near future. Hematoxylin and eosin (H&E) stained histological slides, on the other hand, are routinely generated for a large fraction of cancer patients. Here, we present a deep learning-based approach for multiscale integration of spTx with tumor morphology (MISO). We train MISO to predict spTx from H&E and validate it on a dataset of 72 10X Genomics Visium samples. We further validate our approach on 348 samples from five indications from the MOSAIC consortium and show that MISO significantly outperforms competing methods in extensive benchmarks. We demonstrate that MISO enables near single-cell-resolution, spatially-resolved gene expression prediction.