Adaptive Multi-View and Temporal Fusing Transformer for 3D Human Pose Estimation.

Area of Science:

• Computer Vision
• Machine Learning
• Robotics

Background:

• 3D Human Pose Estimation (HPE) is crucial for various applications.
• Existing methods often require camera calibration and struggle with dynamic scenes.
• Adapting HPE to varying numbers of views and video lengths remains a challenge.

Purpose of the Study:

• To propose a unified framework for adaptive 3D HPE.
• To eliminate the need for camera calibration.
• To handle varying view numbers and video lengths robustly.

Main Methods:

• Developed the Multi-view and Temporal Fusing Transformer (MTF-Transformer) framework.
• Employed a Feature Extractor for 2D pose estimation and feature embedding.
• Introduced a Multi-view Fusing Transformer (MFT) with Relative-Attention for view fusion.
• Utilized a Temporal Fusing Transformer (TFT) for sequence aggregation and 3D pose prediction.

Main Results:

• The MTF-Transformer adaptively fuses multi-view features and temporal information.
• The model achieves competitive quantitative and qualitative results on Human3.6M, TotalCapture, and KTH Multiview Football II datasets.
• Demonstrated generalization to dynamic capture with arbitrary unseen views.

Conclusions:

• The proposed MTF-Transformer framework offers a robust solution for 3D HPE.
• The adaptive nature of the model allows for flexible application in diverse scenarios.
• The method achieves state-of-the-art performance without relying on camera parameters.