PRAFNet: a polarization-RGB adaptive fusion network for object detection in complex environments

Polarization information can enhance object detection performance in complex environments. However, existing polarization fusion strategies are relatively inflexible, limiting the effective utilization of the advantages of the RGB and polarization modalities. To overcome this challenge, a novel (to our knowledge) object detection network named PRAFNet is proposed for robust fusion of polarization and RGB information in complex environments. PRAFNet introduces a flexible adaptive fusion mechanism to maximize the benefits of both modalities. Specifically, the dual-polarization adaptive fusion module (DPAFM) is proposed to generate comprehensive polarization features (Pol), and the polarization-RGB fusion module (PRFM) is proposed to adjust feature weights and couple RGB and polarization features. Meanwhile, the cross-modal differential complementary module (CMDCM) is designed to extract modality-specific strengths at different feature levels, dynamically compensating for their shortcomings. Additionally, the edge-guided polarization enhancement module (EGPEM) is proposed to extract fine details and semantic features from polarization images, further enhancing the utilization of polarization features. Extensive experiments on two public datasets and a self-collected real-world dataset demonstrate that polarization and RGB information can be efficiently fused by PRAFNet, which outperforms existing state-of-the-art detection methods under various complex environments.