Multispectral remote sensing object detection via selective cross-modal interaction and aggregation.

Area of Science:

• Geoscience and Remote Sensing
• Computer Vision
• Artificial Intelligence

Background:

• Multispectral remote sensing object detection is crucial for environmental and disaster monitoring.
• Effective fusion of RGB and infrared data is key for system performance.
• Challenges include capturing cross-modal dependencies and suppressing noise during fusion.

Purpose of the Study:

• To propose a novel framework, Selective cross-modal Interaction and Aggregation (SIA), for improved multispectral remote sensing object detection.
• To enhance the capture of meaningful cross-modal long-range dependencies.
• To suppress noise and irrelevant information during feature fusion for better discriminative quality.

Main Methods:

• The proposed SIA framework consists of two components: Selective Cross-modal Interaction (SCI) and Selective Feature Aggregation (SFA) modules.
• The SCI module selectively prioritizes informative long-range dependencies, reducing computational costs.
• The SFA module uses a gating mechanism to filter noise and redundant information from feature fusion.

Main Results:

• The SIA framework achieved superior detection accuracy on the DroneVehicle, M³FD, and LLVIP datasets.
• On the DroneVehicle benchmark, the proposed method outperformed C²Former by 2.8% mAP@0.5.
• The approach demonstrated lower computational costs compared to existing methods.

Conclusions:

• The SIA framework effectively addresses challenges in multispectral remote sensing object detection by improving feature fusion.
• Selective interaction and aggregation lead to higher accuracy and efficiency.
• The method shows significant potential for various geoscience and remote sensing applications.