Adaptive Fine-Grained Fusion Network for Multimodal UAV Object Detection

Area of Science:

• Computer Vision
• Artificial Intelligence
• Robotics

Background:

• Multimodal perception is crucial for unmanned aerial vehicle (UAV) object detection.
• Global fusion strategies in existing methods struggle with illumination variations and occlusions common in UAV imagery.
• These limitations lead to suboptimal performance in dense, small object detection scenarios.

Purpose of the Study:

• To develop an adaptive, fine-grained fusion network for enhanced multimodal UAV object detection.
• To address the limitations of global fusion by considering local feature consistency and modality-specific information.

Main Methods:

• Proposed an adaptive fine-grained fusion network for multimodal UAV object detection.
• Introduced a local feature consistency-based modality fusion module to adaptively assign fusion weights.
• Implemented a mutual information-guided feature contrastive loss to preserve modality-specific information during early training.

Main Results:

• The proposed method effectively handles object occlusion in UAV perspectives.
• Achieved state-of-the-art performance on multimodal UAV object detection benchmarks.
• Demonstrated superior feature aggregation through adaptive local fusion.

Conclusions:

• The adaptive fine-grained fusion network offers a significant advancement in multimodal UAV object detection.
• The method's ability to handle varying illumination and occlusions makes it robust for real-world UAV applications.
• Future work may involve exploring more sophisticated fusion strategies and attention mechanisms.