MamWorld for open-world object detection with state space modeling and cross-modal fusion

Area of Science:

• Computer Vision
• Artificial Intelligence
• Robotics

Background:

• Unmanned aerial vehicle (UAV) perception requires detecting diverse objects, including known, unseen, and rare categories, in challenging aerial scenes.
• Existing detectors struggle with long-range dependencies, computational complexity, and shallow vision-language fusion, limiting open-category detection accuracy.
• Small targets, cluttered backgrounds, and viewpoint variations pose significant challenges for UAV-based object detection.

Purpose of the Study:

• To introduce MamWorld, a unified state-space-based detection framework for UAV-oriented open-category object detection.
• To improve the modeling of long-range dependencies and enhance semantic consistency between aerial imagery and textual descriptions.
• To enable efficient and accurate detection of known, unseen, long-tail, and unknown objects in complex aerial environments.

Main Methods:

• Developed an ODMamba-based backbone with MassBlock for enhanced cross-channel interaction and multi-scale spatial aggregation.
• Employed TextMambaBlock and SGSS-TextMambaBlock for semantically selective textual representation generation.
• Implemented MambaFusion-PAN for recursive bidirectional vision-language fusion across multiple scales.

Main Results:

• MamWorld demonstrates strong performance on LVIS, M-OWODB, and S-OWODB datasets, particularly for rare and unknown object categories.
• The framework achieves efficient alignment between UAV imagery and open-category semantics.
• Maintained practical efficiency suitable for onboard deployment in UAV perception scenarios.

Conclusions:

• MamWorld offers a unified and efficient solution for open-category object detection in UAVs.
• The state-space-based approach effectively addresses limitations of existing convolutional and Transformer-based detectors.
• The proposed method shows significant potential for advancing UAV perception capabilities in complex, open-world scenarios.