Deeply Coupled Convolution-Transformer With Spatial-Temporal Complementary Learning for Video-Based Person...

Area of Science:

• Computer Vision
• Artificial Intelligence
• Machine Learning

Background:

• Deep convolutional neural networks (CNNs) excel in video-based person re-identification (Re-ID) but have limited global representation.
• Transformers offer global observations by exploring inter-patch relationships, improving performance.
• There's a need to combine the strengths of both CNNs and Transformers for advanced Re-ID.

Purpose of the Study:

• To propose a novel spatial-temporal complementary learning framework, Deeply Coupled Convolution-Transformer (DCCT), for high-performance video-based person Re-ID.
• To leverage the complementary nature of features extracted by CNNs and Transformers.
• To enhance both spatial and temporal feature learning for more robust person Re-ID.

Main Methods:

• Coupling CNNs and Transformers to extract distinct visual features and verifying their complementarity.
• Implementing Complementary Content Attention (CCA) for spatial complementary learning.
• Utilizing Hierarchical Temporal Aggregation (HTA) and Gated Attention (GA) for temporal complementary learning.
• Employing a self-distillation training strategy for knowledge transfer and efficiency.

Main Results:

• The proposed DCCT framework successfully integrates CNN and Transformer features for more informative representations.
• Experimental results on four public Re-ID benchmarks show superior performance compared to state-of-the-art methods.
• The framework demonstrates enhanced accuracy and efficiency in video-based person Re-ID tasks.

Conclusions:

• The DCCT framework effectively addresses the limitations of individual CNN and Transformer models in video-based person Re-ID.
• The spatial-temporal complementary learning approach significantly improves Re-ID performance.
• This work provides a promising direction for future research in person re-identification.