A 3D Point Cloud Gesture Estimation Method Based on EdgeConv Reconstruction of Joint Features

Area of Science:

• Computer Vision
• Human-Computer Interaction
• Machine Learning

Background:

• Gesture interaction offers natural human-computer interfaces but faces challenges like frequent hand movements and self-occlusion.
• Existing 3D point cloud gesture estimation methods lack the accuracy needed for seamless interaction.
• Hand joint self-occlusion and viewpoint variations degrade the performance of current gesture recognition systems.

Purpose of the Study:

• To propose a novel 3D point cloud gesture estimation method to improve accuracy and robustness.
• To leverage EdgeConv for effective reconstruction of hand joint features.
• To enhance natural gesture interaction through precise 3D hand pose estimation.

Main Methods:

• Converting depth maps to 3D point clouds to reduce viewpoint variations.
• Utilizing EdgeConv for initial joint estimation and global feature reconstruction, capturing structural information between joints.
• Performing local joint refinement using EdgeConv twice to incorporate reference and structural information within local point groups.

Main Results:

• The proposed method demonstrates high accuracy, generalization, and robustness across multiple 3D point cloud datasets (ICVL, NYU, MSRA) and complex scenes (InterHand2.6M).
• EdgeConv effectively enriches hand joint feature information, significantly improving estimation performance.
• Successful development of virtual-real interaction applications validated the method's practical utility.

Conclusions:

• The EdgeConv-based 3D point cloud gesture estimation method significantly advances the accuracy and reliability of hand pose recognition.
• This improved gesture estimation provides a strong foundation for developing more intuitive and natural human-computer interaction systems.
• The method's robustness and generalization capabilities make it suitable for diverse real-world gesture interaction scenarios.