A Sampling Approach to Generating Closely Interacting 3D Pose-Pairs from 2D Annotations.

Area of Science:

• Computer Vision
• Human Pose Estimation
• Machine Learning

Background:

• Acquiring 3D human pose data for close interactions is challenging.
• Existing methods often rely on difficult 3D sensor acquisition or direct 2D-to-3D lifting.
• Abundant video data exists but is underutilized for generating interacting 3D poses.

Purpose of the Study:

• To develop a data-driven method for generating numerous plausible, closely interacting 3D human pose-pairs.
• To leverage existing video data for synthesizing complex human interactions.
• To overcome the limitations of traditional 3D pose reconstruction methods.

Main Methods:

• Utilized Markov Chain Monte Carlo (MCMC) sampling, specifically the Metropolis-Hastings algorithm.
• Developed a novel representation called interaction coordinates (IC) for integrated pose and interaction encoding.
• Defined a probability density function (PDF) to guide sampling towards physically valid and plausible 3D pose-pairs.

Main Results:

• Successfully generated a large volume of plausible, closely interacting 3D human pose-pairs.
• Demonstrated efficient sampling over the space of close interactions.
• Showcased good coverage of input 2D pose-pairs from video data.

Conclusions:

• The proposed MCMC-based sampling method effectively synthesizes realistic 3D human pose-pairs for close interactions.
• The novel interaction coordinates representation aids in defining and achieving plausible human interactions.
• This approach offers a viable solution for data augmentation in human motion analysis.