Deciphering spatial heterogeneity by multimodal spatial transcriptomics modelling with SpatialModal

Advances in spatial transcriptomics (ST) technologies have made it possible to jointly acquire gene expression and histological image information while preserving spatial coordinates. This breakthrough presents unprecedented opportunities for the precise dissection of spatial heterogeneity in complex tissues. However, existing computational methods remain limited in their capacity for effective integration and synergistic modelling of multimodal ST data.

We propose SpatialModal, a multimodal graph learning framework that learns robust joint representations by combining a hierarchical representation strategy with a dual-level contrastive learning mechanism. We perform extensive validation of SpatialModal across diverse ST datasets spanning human and mouse tissues. The results demonstrate that SpatialModal effectively reveals intricate brain architectures in humans and mice, dissects tumour microenvironment heterogeneity in breast cancer, delineates Alzheimer's disease patterns, and characterizes spatiotemporal developmental trajectories within the embryonic heart, underscoring its capability to decipher the spatial heterogeneity of biological tissues. Furthermore, SpatialModal exhibits remarkable versatility and robustness, maintaining superior efficacy even on unimodal datasets devoid of histological images, thereby ensuring its broad applicability across diverse ST platforms.

SpatialModal is implemented in Python and is freely available at https://github.com/xingyili/SpatialModal. The source code used in this study has been archived on Zenodo at DOI: https://doi.org/10.5281/zenodo.21264356. All datasets used in this study are publicly available at https://doi.org/10.5281/zenodo.18220735.