S²ANet: Semantic-spatial driven alignment salient object detection network in UAV-based unregistered RGB-T image

Area of Science:

• Computer Vision
• Artificial Intelligence
• Robotics

Background:

• Ground-based systems struggle with salient object detection in complex traffic environments.
• Unmanned aerial vehicles (UAVs) offer flexible perspectives and complementary RGB and thermal imaging capabilities.
• Direct fusion of RGB and thermal data can lead to artifacts due to spatial misalignment and semantic discrepancies.

Purpose of the Study:

• To propose a novel network, the Semantic-Spatial Driven Alignment Network (S²ANet), for salient object detection in UAV-based unregistered RGB-T imagery.
• To address challenges of spatial misalignment and semantic discrepancies in cross-modal fusion for UAV imagery.
• To achieve precise cross-modal registration and enhance feature representation for improved detection.

Main Methods:

• Developed a Semantic-Spatial Alignment (SSA) module for progressive semantic and spatial alignment, enabling precise cross-modal registration.
• Introduced specialized components: Deep Positional Awareness (DPA) for positional cues via self-attention, Cross-Hierarchical Contextual Interaction (CCI) for feature dependencies, and Multi-scale Detail Perception (MDP) for fine-grained details.
• Integrated features using a Dual-Cascade Feature Fusion (DFF) module to generate high-quality saliency maps.

Main Results:

• The S²ANet achieved accurate feature alignment between RGB and thermal modalities.
• Demonstrated superior salient object detection performance in challenging UAV-based RGB-T scenarios.
• Validated the effectiveness of the proposed SSA, DPA, CCI, MDP, and DFF modules in enhancing detection.

Conclusions:

• S²ANet effectively overcomes challenges in UAV-based RGB-T salient object detection by enabling precise cross-modal alignment.
• The network's specialized modules significantly enhance feature representation and fusion, leading to high-quality saliency maps.
• S²ANet offers a robust solution for salient object detection in complex traffic environments using UAVs.