DyCrowd: Towards Dynamic Crowd Reconstruction from a Large-scene Video.

Area of Science:

• Computer Vision
• 3D Reconstruction
• Human Motion Analysis

Background:

• Current 3D crowd reconstruction methods lack temporal consistency and struggle with occlusions.
• Reconstructing dynamic crowds in large scenes is crucial for applications like city surveillance and crowd analysis.

Purpose of the Study:

• To introduce DyCrowd, the first framework for spatio-temporally consistent 3D reconstruction of dynamic crowds in large scenes.
• To address challenges of temporal instability and severe occlusions in crowd reconstruction.

Main Methods:

• A coarse-to-fine group-guided motion optimization strategy for occlusion-robust reconstruction.
• Incorporation of a Variational Autoencoder (VAE)-based human motion prior and segment-level group-guided optimization.
• Leveraging collective crowd behavior and Asynchronous Motion Consistency (AMC) loss for robust motion recovery.

Main Results:

• DyCrowd achieves state-of-the-art performance in large-scene dynamic crowd reconstruction.
• The method enables high-quality motion recovery even with temporal desynchronization and rhythmic inconsistencies.
• Successful reconstruction of hundreds of individuals' poses, positions, and shapes from video.

Conclusions:

• DyCrowd provides a robust solution for spatio-temporally consistent 3D crowd reconstruction in large scenes.
• The proposed methods effectively handle occlusions and temporal inconsistencies using collective crowd dynamics.
• The VirtualCrowd dataset facilitates future research in dynamic crowd analysis.