End-to-End Network for Pedestrian Detection, Tracking and Re-Identification in Real-Time Surveillance System

Surveillance video has been widely used in business, security, search, and other fields. Identifying and locating specific pedestrians in surveillance video has an important application value in criminal investigation, search and rescue, etc. However, the requirements for real-time capturing and accuracy are high for these applications. It is essential to build a complete and smooth system to combine pedestrian detection, tracking and re-identification to achieve the goal of maximizing efficiency by balancing real-time capture and accuracy. This paper combined the detector and Re-ID models into a single end-to-end network by introducing a new track branch to YOLOv5 architecture for tracking. For pedestrian detection, we employed the weighted bi-directional feature pyramid network (BiFPN) to enhance the network structure based on the YOLOv5-Lite, which is able to further improve the ability of feature extraction. For tracking, based on Deepsort, this paper enhanced the tracker, which uses the Noise Scale Adaptive (NSA) Kalman filter to track, and adds adaptive noise to strengthen the anti-interference of the tracking model. In addition, the matching strategy is further updated. For pedestrian re-identification, the network structure of Fastreid was modified, which can increase the feature extraction speed of the improved algorithm by leaps and bounds. Using the proposed unified network, the parameters of the entire model can be trained in an end-to-end method with the multi-loss function, which has been demonstrated to be quite valuable in some other recent works. Experimental results demonstrate that pedestrians detection can obtain a 97% mean Average Precision (mAP) and that it can track the pedestrians well with a 98.3% MOTA and a 99.8% MOTP on the MOT16 dataset; furthermore, high pedestrian re-identification performance can be achieved on the VERI-Wild dataset with a 77.3% mAP. The overall framework proposed in this paper has remarkable performance in terms of the precise localization and real-time detection of specific pedestrians across time, regions, and cameras.