Remembering What Is Important: A Factorised Multi-Head Retrieval and Auxiliary Memory Stabilisation Scheme for Human...

Area of Science:

• Computer Vision
• Machine Learning
• Artificial Intelligence

Background:

• Human motion forecasting is complex due to variations in activity, interaction, and individual preferences.
• Accurate prediction of future human poses requires effective modeling of historical motion data.

Purpose of the Study:

• To develop an innovative deep neural network framework with auxiliary memory for enhanced human motion forecasting.
• To improve the modeling of historical knowledge by disentangling various factors from observed pose sequences.

Main Methods:

• A novel auxiliary-memory-powered deep neural network framework is proposed.
• Factorized features (subject-specific, action-specific, auxiliary) are used to query an auxiliary memory.
• A Multi-Head knowledge retrieval scheme and dynamic masking strategy are employed for adaptive feature disentanglement.
• Two novel loss functions encourage memory diversity and stability for salient information storage.

Main Results:

• The proposed framework significantly outperforms state-of-the-art methods on Human3.6M (17% improvement) and CMU-Mocap (9% improvement) datasets.
• The design choices collectively enhance the accuracy and robustness of long-term human motion prediction.
• The method demonstrates effectiveness across data imbalances and diverse input distributions.

Conclusions:

• The auxiliary-memory deep neural network framework offers a significant advancement in human motion forecasting.
• Effective disentanglement and retrieval of historical motion information are crucial for accurate predictions.
• The proposed approach provides a robust solution for complex human motion prediction tasks.