A variational graph autoencoder guided by residual multi-head attention reveals heterogeneity in spatial...

Spatial transcriptomics has recently emerged as a crucial technical tool for uncovering tissue structural heterogeneity and cellular spatial relationships, as it simultaneously captures gene expression data and spatial location information. However, current methods still have limitations in multimodal information integration, local boundary delineation, and the modeling of complex spatial dependencies, making it difficult to adequately capture the spatial structural features of tissues where local and global characteristics coexist. To this end, this paper proposes RMVGAE, a spatial transcriptomics analysis model that combines residual multi-head attention mechanisms with variational graph autoencoders. The model takes histological images, gene expression matrices, and spatial location information as inputs. It constructs morphological similarity matrices, gene similarity matrices, and spatial adjacency matrices, achieving enhanced multimodal representations through neighborhood feature fusion. Critical features are captured by introducing a residual multi-head attention module, and latent representations of spatial spots are learned using a variational graph autoencoder. The experimental results indicate that RMVGAE is capable of identifying spatial domains more accurately across multiple spatial transcriptomic datasets. Differential expression analysis and enrichment analysis further demonstrate that this model has significant biological relevance and application potential. Ablation experiments also validated the effectiveness of the designed residual multi-head attention module in spatial domain recognition.