A 3D Hand Attitude Estimation Method for Fixed Hand Posture Based on Dual-View RGB Images

Area of Science:

• Computer Vision
• Robotics
• Machine Learning

Background:

• Teleoperation requires accurate 3D hand pose estimation.
• Hand self-occlusion is a significant challenge in 3D hand posture analysis.
• Existing methods often require complex sensor setups.

Purpose of the Study:

• To develop a robust 3D hand attitude estimation method using dual RGB cameras.
• To enable teleoperation applications by overcoming hand self-occlusion.
• To create an efficient dataset acquisition pipeline for 3D hand posture.

Main Methods:

• A novel approach combining Convolutional Neural Networks (CNNs) and LightGBM for 3D hand attitude estimation.
• Utilizing dual-view RGB images and an IMU sensor for dataset creation with automatic attitude angle labeling.
• An integrated CNN regression model with Bayesian optimization-based LightGBM for ensemble learning.

Main Results:

• The proposed method successfully estimates 3D hand attitude from dual RGB images.
• Demonstrated effectiveness in solving the hand self-occlusion problem.
• Achieved accurate 3D hand attitude estimation using only two standard RGB cameras.

Conclusions:

• The developed method provides a practical solution for 3D hand attitude estimation in teleoperation.
• Dual-view RGB imaging combined with advanced machine learning offers a viable alternative to complex sensors.
• This approach enhances the feasibility of intuitive human-robot interaction.