Transformer-Based Model with Dynamic Attention Pyramid Head for Semantic Segmentation of VHR Remote Sensing Imagery.

Area of Science:

• Computer Vision
• Remote Sensing
• Machine Learning

Background:

• Convolutional neural networks (CNNs) face limitations in capturing long-range dependencies in very-high-resolution (VHR) remote sensing (RS) images due to fixed receptive fields.
• Swin Transformer excels at modeling long-range dependencies but struggles with intra-patch positional information loss.

Purpose of the Study:

• To propose SUD-Net (Swin transformer-based U-Net-like with Dynamic attention pyramid head Network), a hybrid architecture for improved semantic segmentation of VHR RS images.
• To overcome the limitations of existing CNN and transformer models in capturing both global context and local details.

Main Methods:

• A dual-encoder architecture integrating Swin Transformer blocks and ResNet layers to capture global semantics and detailed representations.
• A Multi-Path Fusion Model (MPFM) with Patch Attention (PA) to address spatial information loss within image patches and fuse multi-scale features.
• A Dynamic Attention Pyramid Head (DAPH) utilizing deformable convolution for adaptive aggregation of semantic information.

Main Results:

• SUD-Net achieved state-of-the-art results on the ISPRS Potsdam dataset, reaching 92.51% mF1-score, 86.4% mIoU, and 92.98% Overall Accuracy (OA).
• On the ISPRS Vaihingen dataset, SUD-Net obtained 89.49% mF1-score, 81.26% mIoU, and 90.95% OA.
• The proposed architecture effectively integrates global context and local details for enhanced semantic segmentation.

Conclusions:

• SUD-Net demonstrates the efficacy of combining Swin Transformer and U-Net principles for VHR RS image semantic segmentation.
• The novel components, MPFM with PA and DAPH, successfully address key challenges in patch-based processing and feature aggregation.
• The superior performance on benchmark datasets validates SUD-Net as a powerful tool for remote sensing image analysis.