Multi-Granularity Aggregation with Spatiotemporal Consistency for Video-Based Person Re-Identification

Area of Science:

• Computer Vision
• Artificial Intelligence
• Machine Learning

Background:

• Video-based person re-identification (ReID) relies on spatial and temporal features.
• Existing methods struggle with frame inconsistencies caused by occlusion and detection errors.
• These inconsistencies hinder effective temporal processing and spatial information balance.

Purpose of the Study:

• To propose a novel method, Spatiotemporal Multi-Granularity Aggregation (ST-MGA), for robust video-based person ReID.
• To address feature inconsistencies and enhance spatiotemporal information aggregation.
• To improve the accuracy and efficiency of person re-identification in videos.

Main Methods:

• Developed the ST-MGA framework with extraction, augmentation, and aggregation stages.
• Introduced the consistent part-attention (CPA) module for spatiotemporally aligned part extraction.
• Incorporated Multi-Attention Part Augmentation (MA-PA) and Long-/Short-term Temporal Augmentation (LS-TA) blocks for diverse feature capture.

Main Results:

• The CPA module extracts consistent and well-aligned parts, mitigating misalignment issues.
• MA-PA and LS-TA blocks enhance spatial and temporal feature diversity.
• ST-MGA effectively aggregates multi-granular spatiotemporal patterns by analyzing part relations and scales.

Conclusions:

• ST-MGA demonstrates state-of-the-art performance on MARS, DukeMTMC-VideoReID, and LS-VID benchmarks.
• The method successfully overcomes challenges posed by occlusion and imperfect detection in video ReID.
• ST-MGA offers a significant advancement in video-based person re-identification by leveraging consistent spatiotemporal cues.