Self-Supervised Multi-View Person Association and its Applications.

Area of Science:

• Computer Vision
• Machine Learning
• Human Motion Analysis

Background:

• Markerless motion tracking in complex group activities is difficult due to occlusions, viewpoint changes, and asynchronous video.
• Accurate person association across different views and times is crucial for reliable tracking.

Purpose of the Study:

• To develop a self-supervised framework for adapting person appearance descriptors to unlabeled videos.
• To improve markerless motion tracking and 3D human skeleton reconstruction in challenging group scenarios.

Main Methods:

• A self-supervised framework adapts generic person appearance descriptors using motion tracking, mutual exclusion, and multi-view geometry.
• The adapted descriptor is applied in a tracking-by-clustering approach.
• Reconstructed 3D skeletons are used to create multi-angle videos, switching cameras based on occlusion detection.

Main Results:

• Significant improvements in association accuracy (up to 18%) and 3D human skeleton tracking (5-10 times) over baseline methods.
• Effective handling of occlusions for seamless camera switching in multi-angle video generation.
• Validation on the WILDTRACK dataset and three new complex social scenes with up to 60 people.

Conclusions:

• The proposed self-supervised framework enhances person association and 3D skeleton tracking in complex, multi-camera scenarios.
• The method provides a robust solution for markerless motion tracking, outperforming existing approaches.
• The system enables dynamic multi-angle video generation by intelligently switching cameras based on occlusion awareness.