Semantic-spatial synergy feature pyramid network for dense tiny object detection from UAV images

Different from general object detection, unmanned aerial vehicle (UAV) images include a large number of densely distributed and tiny-scale objects due to the changes in shooting angles and altitudes. Feature pyramid network (FPN), as one of the most important components of object detection models, has greatly improved the performance of object detection. However, when dealing with objects from UAV images, FPN tends to weaken the crucial spatial features of objects of interest, hindering performance in accurately locating objects. To overcome the above issues, we propose a novel semantic-spatial synergy feature pyramid network, named S³-FPN for dense tiny object detection in UAV images. Specifically, the newly proposed S³-FPN consists of four major parts. Firstly, the cross-channel compression module (CCM) interacts with two groups of channel separation features, lowering the computational burden of the network. Secondly, the top-down semantic perception module (TSPM) and another bottom-up spatial perception module (BSPM) form two independent branches, which complement and enhance the semantic and spatial features at different stages respectively. Thirdly, the hierarchical kernel interaction module (HKIM) is elaborately formulated to integrate multi-scale features to compensate for the tiny object features that might be lost in the deeper layers. Finally, the enhanced semantic and spatial features at the same stage are integrated for object classification and localization. To validate the effectiveness of the proposed S³-FPN, extensive experiments on four datasets with densely distributed tiny objects, i.e., TinyPerson, visDrone, DOTA-v1.0 and HIT-UAV datasets indicate that the S³-FPN achieves competitive performance compared to other state-of-the-art (SOTA) FPN models. The code will be available at https://github.com/D-ace11/.