HESpotEx: a dual-stream deep learning framework for spot-level gene expression prediction from histological images

Area of Science:

• Computational pathology
• Genomics
• Bioinformatics

Background:

• Whole-slide images (WSIs) are crucial for disease diagnosis and prognosis.
• Spatial transcriptomics (ST) reveals gene expression but is expensive.
• Integrating WSI and ST data is challenging but valuable.

Purpose of the Study:

• To develop a cost-effective method for predicting spatial gene expression from WSIs.
• To create a deep learning framework (HESpotEx) for this prediction task.
• To leverage multimodal data for enhanced histopathological analysis.

Main Methods:

• HESpotEx: A dual-stream multimodal deep learning framework.
• Utilizes graph attention autoencoders, an image encoder, and a graph convolution network decoder.
• Predicts expression for up to 5,457 genes from WSI data.

Main Results:

• HESpotEx accurately predicts spatial gene expression patterns from WSIs.
• Demonstrates superior performance and robustness across diverse ST datasets (cancer and non-cancer).
• Identifies diagnosis-associated WSI patches and shows improved cross-sectional consistency on high-resolution ST data.

Conclusions:

• HESpotEx offers a cost-effective alternative to ST for spatial gene expression analysis.
• The framework can decipher spatial molecular characteristics from histological patterns.
• Potential to significantly advance disease diagnosis and prognosis through integrated WSI and molecular data.