Learning to Estimate External Forces of Human Motion in Video

Area of Science:

• Biomechanics
• Computer Vision
• Machine Learning

Background:

• Accurate measurement of ground reaction forces (GRFs) is crucial for sports performance analysis and injury prevention.
• Traditional methods using force plates are costly, time-consuming, and lack experimental consistency.
• Existing video-based methods using LSTMs have limitations in modeling capacity.

Purpose of the Study:

• To propose a novel transformer-based approach for inferring GRFs from video data.
• To introduce a new loss function for minimizing impact peaks in GRF estimations.
• To enhance model generalization through pre-training and multi-task learning.

Main Methods:

• Utilized a transformer architecture for GRF inference from video, a first in this domain.
• Developed a novel loss function to specifically address high impact peaks in predicted GRF curves.
• Implemented pre-training and multi-task learning strategies using 2D-to-3D human pose estimation.
• Evaluated the model on the LAAS Parkour dataset and the new ForcePose dataset.

Main Results:

• Achieved up to a 19% reduction in error compared to previous state-of-the-art methods.
• Demonstrated improved generalization to unseen motions through pose estimation pre-training.
• Showcased the benefit of pre-training for fine-tuning on smaller GRF datasets.

Conclusions:

• The proposed transformer-based method offers a more accurate and efficient alternative for GRF estimation from video.
• Pre-training on human pose estimation significantly enhances the model's ability to generalize and perform on limited data.
• This approach has the potential to make GRF analysis more accessible for sports science and injury prevention research.