Visual spatial relationship sensitive transformer for image captioning

Image captioning is a cross-modal task that combines computer vision and natural language processing to generate natural language descriptions of visual content. Recent advances have explored the integration of both grid-based and region-based visual features to better capture relational and contextual information, such as interactions between objects and their surrounding environment. However, combining multiple types of features often results in spatial misalignment, which hinders the model's ability to construct coherent visual-semantic relationships. To address these limitations, we propose a novel Spatial Alignment Positional Encoder(SAPE), which encodes spatial information across aligned grid-level and region-level features to construct a unified visual-spatial representation. In addition, we introduce two complementary enhancement modules: Group Normalization Multi-head Attention(GNMA) to capture global relational cues, and Convolution-based Feature Enhancement Attention(CFEA) to enrich local spatial details. To mitigate the degradation of positional signals during deep training, we further propose a Learnable Adaptive Positional Encoder(LAPE) that dynamically preserves position-sensitive information. These components are integrated into a unified transformer-based architecture named the Visual-Spatial Relationship Sensitive Transformer(VRST). Extensive experiments on the MSCOCO dataset demonstrate the effectiveness of our approach, achieving a CIDEr score of 141.9 on the Karpathy test split and 138.2 on the official evaluation server, surpassing several strong baselines.