Building digital histology models of transcriptional tumor programs with generative deep learning for pathology-based...

Precision oncology depends on identifying the biological vulnerabilities of a tumor. Molecular assays, like transcriptomics, provide an information-rich view of the tumor that can be leveraged to inform therapeutic selection. However, the costs of such assays can be prohibitive for clinical translation at scale. Histology-based imaging remains a predominant means of diagnosis that is widely accessible. To more broadly leverage limited molecular datasets, models have been trained to use histology to infer the expression of individual genes or pathways, with varying levels of accuracy and explainability.

Our approach detects expression of transcriptional programs from tumor histology and interprets the image features supporting program detection. Specifically, we used RNA-seq data from squamous cell carcinoma (SCC) patients to infer cohesive expression patterns of multiple genes. Then, we used deep learning techniques to train a computational model to predict the activity levels of the transcriptional programs directly from histology images. We exploited that predictive capability to generate synthetic digital models of the cellular histology of each transcriptional program, using generative adversarial networks to isolate image features supporting specific transcriptional predictions and pathologist review to interpret the images.

Applying our histologically integrated latent space analysis to SCCs revealed sets of genes associated with both pathologist-interpretable image features and clinically relevant processes, including immune response, collagen remodeling, and fibrosis, going beyond predictions of individual molecular features.

Our results demonstrate an approach for discovering clinically interpretable histological features that indicate molecular, potentially treatment-informing, biological processes. These features are detectable in widely available histology slides, allowing a standard microscope to deliver complex, patient-specific molecular information.