PoseBERT: A Generic Transformer Module for Temporal 3D Human Modeling.

Area of Science:

• Computer Vision
• Machine Learning
• Robotics

Background:

• Human pose estimation in videos is hindered by the difficulty and cost of acquiring annotated datasets.
• Existing transformer methods for pose sequence modeling often rely on pseudo-ground truth for data augmentation.

Purpose of the Study:

• Introduce PoseBERT, a versatile transformer module for enhancing video-based human pose estimation.
• Enable leveraging temporal information from 3D Motion Capture (MoCap) data for pose modeling.

Main Methods:

• PoseBERT is trained on 3D MoCap data using a masked modeling approach.
• The module is designed to be integrated with existing image-based pose estimation models.
• Variants accept diverse inputs, including 3D skeleton keypoints and parametric model rotations (SMPL, MANO).

Main Results:

• PoseBERT consistently improves the performance of state-of-the-art pose estimation methods.
• The module demonstrates task-agnostic capabilities, applicable to pose refinement, prediction, and completion without finetuning.
• A real-time demo showcases smooth robotic hand animation using PoseBERT with webcam input.

Conclusions:

• PoseBERT offers a simple, versatile, and effective solution for incorporating temporal dynamics into human pose estimation.
• The method significantly advances video-based pose modeling by utilizing readily available MoCap data.
• Its low computational cost and broad applicability make it suitable for real-time applications.