SRGFormer: Semantic Role-Guided Graph Reasoning for Referring Remote Sensing Image Segmentation

Area of Science:

• Computer Science
• Artificial Intelligence
• Remote Sensing

Background:

• Referring remote sensing image segmentation (RRSIS) enables object retrieval and localization in Earth observation using natural language.
• Existing RRSIS methods often entangle semantic cues, hindering precise instance differentiation in complex scenes with distractors.
• Convolution-dominated decoding and holistic language features limit the ability to distinguish targets from similar objects.

Purpose of the Study:

• To develop a novel graph reasoning framework, SRGFormer, to address limitations in current RRSIS methods.
• To improve the accuracy and granularity of instance segmentation in remote sensing imagery based on textual descriptions.
• To enhance the model's ability to distinguish target objects from distractors by explicitly modeling semantic roles.

Main Methods:

• Proposed SRGFormer, a graph reasoning framework for RRSIS.
• Introduced a semantic role decomposition (SRD) module to break down expressions into target, relation, and position semantics.
• Developed a semantic-relational graph transformer (SRGT) for relation-aware graph reasoning and a progressive mask refinement (PMR) module for iterative mask generation.

Main Results:

• SRGFormer achieved state-of-the-art performance on the RefSegRS benchmark, with 66.08% mIoU and 76.93% oIoU.
• Demonstrated significant improvements in precision, with a 15.95% gain in Pr@0.7.
• Showcased competitive performance on the RRSIS-D benchmark (65.87% mIoU, 24.61% Pr@0.9), indicating general applicability.

Conclusions:

• SRGFormer effectively improves target localization and fine-grained mask prediction in complex remote sensing scenes.
• Explicitly modeling semantic roles via SRD and SRGT enhances instance differentiation and reduces semantic fading.
• The proposed framework offers a robust solution for advanced intelligent Earth observation applications.