A wavelet attention fusion and YOLO based approach for robust multi scale hand pose estimation

Area of Science:

• Computer Vision
• Machine Learning
• Human-Computer Interaction

Background:

• 3D hand pose estimation is challenging due to articulation complexity, occlusions, and appearance variations.
• Traditional models struggle with fine details and real-world generalization.
• Accurate 3D hand pose is crucial for robotics, VR, and augmented reality.

Purpose of the Study:

• To develop a novel framework for real-time 3D hand pose estimation.
• To improve accuracy and robustness, especially under occlusion and varying orientations.
• To achieve state-of-the-art performance in precision and inference speed.

Main Methods:

• Integration of a Wavelet Attention Fusion Module (WAFM) with a modified YOLOv11 architecture.
• WAFM uses wavelet decomposition to preserve edge details and enhance semantic features.
• A multi-scale tempo-spatial backbone with Efficient Net and Gated Recurrent Units, combined with a feature fusion neck (FPN/PAN), processes sequential frames.

Main Results:

• The proposed framework significantly enhances accuracy under heavy occlusion and varying orientations.
• Achieved state-of-the-art performance on the Ego Hand and Senz3D datasets.
• Demonstrated real-time inference speed for 3D hand pose reconstruction.

Conclusions:

• The novel framework effectively addresses limitations of traditional 3D hand pose estimation methods.
• The integration of WAFM and YOLOv11 provides a robust solution for complex hand pose scenarios.
• The approach offers a significant advancement for real-time applications requiring precise 3D hand tracking.