Like Human Rethinking: Contour Transformer AutoRegression for Referring Remote Sensing Interpretation

Area of Science:

• Computer Vision
• Remote Sensing
• Artificial Intelligence

Background:

• Referring remote sensing expression comprehension and segmentation (RRSECS) is crucial for ecological protection, resource exploration, and emergency management.
• Existing methods struggle with micro-target localization drift and contour-boundary misalignment in remote sensing imagery.
• Polygon-based methods face challenges in multi-task co-optimization when applied to remote sensing domains.

Purpose of the Study:

• To propose SeeFormer, a novel contour autoregressive paradigm for accurate RRSECS.
• To address challenges in micro-target localization, boundary feature extraction, and contour reconstruction.
• To improve the performance of referring image segmentation and visual grounding in remote sensing.

Main Methods:

• Introduced a brain-inspired feature refocus learning (BIFRL) module for coarse-to-fine feature attention and small-object enhancement.
• Developed a language-contour enhancer (LCE) and corner-based contour sampler (CBCS) for improved shape-aware contour priors and mask-polygon reconstruction.
• Implemented an autoregressive dual-decoder paradigm (ARDDP) to maintain sequence consistency and resolve multi-task optimization conflicts.

Main Results:

• SeeFormer achieved significant performance gains on RefDIOR, RRSIS-D, and OPT-RSVG datasets.
• Outperformed the baseline PolyFormer, improving oIoU by 27.58% and mIoU by 39.37% for referring image segmentation on RefDIOR.
• Achieved 18.94% and 28.90% improvements in oIoU and mIoU for visual grounding on the RefDIOR dataset.

Conclusions:

• SeeFormer offers a transformative solution for RRSECS, accurately locating and segmenting micro, irregular targets.
• The proposed BIFRL, LCE, CBCS, and ARDDP modules effectively address existing limitations in remote sensing interpretation.
• The model demonstrates superior performance, paving the way for advancements in remote sensing applications.