Graph attention automatic encoder based on contrastive learning for domain recognition of spatial transcriptomics

Spatial transcriptomics is an emerging technology that enables the profiling of gene expression in tissues while preserving spatial location information. This innovative approach is anticipated to provide a comprehensive understanding of the spatial distribution of different cells within tissues and facilitate in-depth analysis of tissue structure. To accurately recognize spatial domains from spatial transcriptomics, we have introduced a generalized deep learning method called GAAEST (Graph Attention-based Autoencoder for Spatial Transcriptomics). Our proposed approach effectively integrates both spatial location information and gene expression data from spatial transcriptomics. Specifically, it leverages spatial location details to construct a neighborhood graph and employs a graph attention network-based encoder to embed gene expression information into a spatially informed space. At the same time, to further optimize the learned potential embedding, self-supervised contrastive learning is introduced to capture spatial information at three levels: local, global and contextual feature of spots. Finally, the decoder reconstructs gene expressions, which are then clustered to identify spatial domains with similar expression patterns and spatial proximity. Based on our experiments conducted on multiple datasets, GAAEST consistently outperforms existing state-of-the-art methods. The proposed GAAEST demonstrates excellent capabilities in spatial domain recognition, positioning it as an ideal tool for advancing spatial transcriptomics research.