Integrating histology and spatial transcriptomics via multimodal transformers and contrastive representation learning...

Area of Science:

• Computational Biology
• Genomics
• Medical Imaging

Background:

• Understanding tissue organization and molecular phenotypes relies on predicting spatial gene expression from histological images.
• Current methods struggle with single-model limitations and poor alignment between image and transcriptomic data.

Purpose of the Study:

• To develop a unified multimodal learning framework for integrating histological imaging and spatial transcriptomics.
• To enable effective cross-modal alignment through a shared latent representation space.

Main Methods:

• Utilized a ResNet50 convolutional stem and MobileViT Transformer backbone for histological image encoding.
• Employed linear-GELU-dropout transformation blocks to project modalities into a shared latent space.
• Implemented a contrastive learning objective for cross-modal alignment between image and spot embeddings.

Main Results:

• MViTGene demonstrated significantly higher prediction accuracy compared to existing methods on the human liver Visium dataset.
• Achieved improvements of 20%, 33%, and 12% in predicting marker, highly expressed, and highly variable genes, respectively.
• The model accurately captures the correspondence between tissue morphology and gene expression.

Conclusions:

• MViTGene offers a computational tool for high-throughput spatial gene expression prediction.
• The framework balances performance and interpretability, enabling more reliable biological interpretation.
• Advances the field of spatial transcriptomics analysis through multimodal integration.