Skeleton-Based Human Motion Prediction With Privileged Supervision.

Area of Science:

• Computer Vision
• Machine Learning
• Robotics

Background:

• Supervised methods excel in skeleton-based human motion forecasting but require action labels during both training and inference.
• The necessity of action labels in the inference phase poses practical challenges and limits applicability.
• Incomplete or mixed action labels in training data can hinder model performance.

Purpose of the Study:

• To develop a human motion forecasting model that alleviates the need for action class labels during inference.
• To leverage action class labels as privileged information available only during the training phase.
• To improve the robustness and accuracy of motion prediction models when dealing with incomplete or mixed action labels.

Main Methods:

• A novel neural network architecture integrating motion classification as an auxiliary task alongside motion prediction.
• A new classification loss function designed to exploit relationships within observed and potentially missing motion sequence labels.
• A perceptual loss function to quantify differences between ground truth and generated sequences within the classification task.

Main Results:

• The proposed algorithm demonstrates significant effectiveness in improving human dynamics modeling.
• Experimental validation on challenging datasets like Human 3.6M and CMU datasets confirms the approach's efficacy.
• The method successfully exploits action class labels, even when they are incomplete, for enhanced motion forecasting.

Conclusions:

• Action class labels can be effectively utilized as privileged training information for human motion forecasting.
• The proposed architecture and loss functions provide a robust solution for motion prediction with reduced inference-time label dependency.
• This work advances the field of skeleton-based human motion analysis by offering a more practical and accurate forecasting approach.