QA²FDet: Quality-Aware Adaptive Alignment Fusion Network for UAV RGBT Tiny Pedestrian Detection

Area of Science:

• Computer Vision
• Artificial Intelligence
• Remote Sensing

Background:

• Tiny pedestrian detection in Unmanned Aerial Vehicle (UAV) aerial images is vital for urban security and disaster response.
• Challenges include small pedestrian scale, sparse distribution, background noise, and cross-modal spatial misalignment.

Purpose of the Study:

• To propose QA²FDet, a quality-aware adaptive alignment fusion network to address challenges in visible-thermal tiny pedestrian detection.
• To enhance feature learning and cross-modal fusion for robust detection in aerial imagery.

Main Methods:

• Developed a Quality-Aware Adaptive Alignment Fusion Network (QA²FDet) with three modules: Spectrum-Spatial Decoupled Enhancement (SDE), Cross-Modal Correspondence Mining (CCM), and Prior-Informed Gated Fusion (PGF).
• Utilized Discrete Cosine Transform in SDE to reduce background noise and deep semantic gating for detail enhancement.
• Employed thermal-guided local asymmetric cross-attention in CCM for fine-grained correspondence.
• Integrated region-level quality and modality discrepancy modeling in PGF for adaptive fusion.

Main Results:

• QA²FDet achieved state-of-the-art performance on multiple UAV-based RGBT detection benchmarks.
• Demonstrated strong robustness in challenging aerial scenes with tiny and sparsely distributed pedestrians.

Conclusions:

• QA²FDet effectively overcomes limitations in tiny pedestrian detection by improving feature learning and cross-modal fusion.
• The proposed network shows significant promise for real-world applications in aerial surveillance and emergency response.