Multi-scale semantic enhancement network for object detection

In the field of object detection, feature pyramid network (FPN) can effectively extract multi-scale information. However, the majority of FPN-based methods suffer from a semantic gap between features of various sizes before feature fusion, which can lead to feature maps with significant aliasing. In this paper, we present a novel multi-scale semantic enhancement feature pyramid network (MSE-FPN) which consists of three effective modules: semantic enhancement module, semantic injection module, and gated channel guidance module to alleviate these problems. Specifically, inspired by the strong ability of the self-attention mechanism to model context, we propose a semantic enhancement module to model global context to obtain the global semantic information before feature fusion. Then we propose the semantic injection module to divide and merge global semantic information into feature maps at various scales to narrow the semantic gap between features at different scales and efficiently utilize the semantic information of high-level features. Finally, to mitigate feature aliasing caused by feature fusion, the gated channel guidance module selectively outputs crucial features via a gating unit. By replacing FPN with MSE-FPN in Faster R-CNN, our models achieve 39.4 and 41.2 Average precision (AP) using ResNet50 and ResNet101 as the backbone network respectively. When using ResNet-101-64x4d as the backbone, MSE-FPN achieved up to 43.4 AP. Our results demonstrate that replacing FPN with MSE-FPN significantly enhances the detection performance of state-of-the-art FPN-based detectors.