MuTr: Multi-Stage Transformer for Hand Pose Estimation from Full-Scene Depth Image.

Area of Science:

• Computer Vision
• Machine Learning
• Robotics

Background:

• Accurate hand pose estimation is crucial for human-computer interaction and robotics.
• Existing methods often require separate models for hand localization and pose estimation.

Purpose of the Study:

• To introduce DePOTR, a novel transformer-based method for hand pose estimation.
• To develop MuTr, a multi-stage approach for hand pose estimation from full-scene depth images.
• To demonstrate a unified model architecture for both standard and full-scene image setups.

Main Methods:

• DePOTR: A transformer-based architecture for direct hand pose estimation.
• MuTr: A multi-stage approach integrating hand localization and pose estimation from depth images.
• Evaluation on four benchmark datasets including NYU.

Main Results:

• DePOTR outperforms existing transformer-based methods and matches state-of-the-art performance.
• MuTr achieves competitive results without separate localization and estimation models.
• DePOTR and MuTr reach precisions of 7.85 mm and 8.71 mm on the NYU dataset, respectively.

Conclusions:

• The proposed DePOTR and MuTr methods represent significant advancements in hand pose estimation.
• A single model architecture can effectively handle both standard and full-scene depth image scenarios.
• These methods offer a more streamlined and efficient pipeline for hand pose estimation applications.