Bi-Stream Pose-Guided Region Ensemble Network for Fingertip Localization From Stereo Images

Area of Science:

• Computer Vision
• Human-Computer Interaction
• Robotics

Background:

• Accurate hand pose estimation, particularly fingertip localization, is crucial for human-computer interaction.
• Traditional methods using depth images struggle with noise and missing data.
• Existing stereo-based hand pose datasets have limitations in size, viewpoint, articulation, and shape diversity.

Purpose of the Study:

• To address limitations in current stereo-based hand pose datasets.
• To introduce a new large-scale binocular hand pose dataset, THU-Bi-Hand.
• To propose a novel network, Bi-Pose-REN, for improved fingertip localization.

Main Methods:

• Construction of the THU-Bi-Hand dataset with 447k stereo image pairs and 3D annotations for wrist and fingertips.
• Development of the bi-stream pose-guided region ensemble network (Bi-Pose-REN).
• Extraction of representative feature regions guided by estimated poses and hierarchical integration for refined hand pose regression.

Main Results:

• The THU-Bi-Hand dataset offers extensive coverage of hand shapes, articulations, and viewpoints.
• Bi-Pose-REN demonstrated superior performance in fingertip localization on the THU-Bi-Hand dataset.
• Established benchmarks for evaluating hand pose estimation methods using stereo images.

Conclusions:

• THU-Bi-Hand provides a valuable resource for advancing stereo-based hand pose research.
• Bi-Pose-REN represents a significant improvement in fingertip localization accuracy.
• The proposed methods and dataset facilitate future research in hand pose estimation.