A Dual-Modal Adaptive Pyramid Transformer Algorithm for UAV Cross-Modal Object Detection

Area of Science:

• Computer Vision
• Artificial Intelligence
• Remote Sensing

Background:

• Unmanned Aerial Vehicles (UAVs) are crucial for surveillance and disaster management, requiring reliable infrared-visible image detection.
• Existing UAV detection methods struggle with multi-scale targets, lighting variations, and efficient cross-modal data use.
• Complex illumination conditions pose significant challenges for accurate UAV-based target identification.

Purpose of the Study:

• To develop a lightweight module for enhancing infrared-visible image detection in UAVs.
• To address challenges in multi-scale target recognition and robustness to lighting variations.
• To improve the utilization of cross-modal information for better detection accuracy.

Main Methods:

• Proposed a lightweight Dual-modality Adaptive Pyramid Transformer (DAP) module.
• Integrated the DAP module into the YOLOv8 object detection framework.
• Employed hierarchical self-attention and residual fusion for adaptive multi-scale representation and cross-modal alignment.

Main Results:

• The DAP-based YOLOv8 achieved mAP50:95 scores of 61.2% on DroneVehicle and 62.1% on LLVIP datasets.
• Demonstrated superior performance compared to conventional infrared-visible detection methods.
• Validated the module's effectiveness in complex environments and challenging lighting.

Conclusions:

• The DAP module effectively optimizes cross-modal feature interaction for UAV infrared-visible detection.
• The proposed method offers a practical and efficient solution for real-time UAV applications.
• Enhanced detection accuracy improves UAV capabilities in traffic monitoring and disaster response.