Multi-Task Spatial-Temporal Graph Auto-Encoder for Hand Motion Denoising.

Area of Science:

• Computer Vision
• Human-Computer Interaction
• Machine Learning

Background:

• Accurate hand tracking is essential for immersive human-computer interaction.
• Raw hand motion data often suffers from noise and occlusions, leading to interaction lag.
• Predicting future hand movements is crucial for real-time responsiveness.

Purpose of the Study:

• To develop a novel model for simultaneously denoising and predicting hand motion.
• To improve the accuracy and reduce latency in hand tracking for HCI applications.
• To leverage the inter-dependency between denoising and prediction tasks for enhanced performance.

Main Methods:

• Introduced the Multi-task Spatial-Temporal Graph Auto-Encoder (Multi-STGAE) model.
• Integrated a gate mechanism to prevent negative transfer between tasks.
• Utilized spatial-temporal graph autoencoder blocks with graph convolutional networks for motion modeling.
• Developed a novel hand partition strategy and hand bone loss for natural motion generation.

Main Results:

• The Multi-STGAE model effectively denoises and predicts hand motion, outperforming state-of-the-art methods.
• The multi-task framework demonstrated mutual benefits between denoising and prediction.
• Introduced two large-scale datasets and two structural metrics for evaluating hand motion naturalness.
• Validated the model's ability to maintain motion dynamics and avoid over-smoothed results.

Conclusions:

• The Multi-STGAE model offers a robust solution for accurate and responsive hand tracking in HCI.
• The proposed multi-task learning approach significantly enhances both denoising and prediction capabilities.
• The method contributes to more natural and immersive human-computer interaction experiences.