Visual-inertial hand motion tracking with robustness against occlusion, interference, and contact

Area of Science:

• Robotics
• Human-Computer Interaction
• Computer Vision

Background:

• Current hand motion tracking technologies face limitations like occlusions, electromagnetic interference, and sensor ambiguity.
• Existing methods often lack accuracy and robustness in real-world scenarios.

Purpose of the Study:

• To develop a robust and accurate hand and finger motion tracking framework.
• To address the limitations of current vision, inertial, and soft sensor tracking methods.

Main Methods:

• A novel visual-inertial skeleton tracking (VIST) framework was developed.
• The framework utilizes a sensor glove with inertial measurement units (IMUs) and visual markers, paired with a stereo camera.
• A tightly coupled filtering-based visual-inertial fusion algorithm estimates motion and autocalibrates kinematic parameters.

Main Results:

• The VIST framework demonstrates good tracking accuracy and robustness across diverse real-world conditions.
• The system is affordable, lightweight, and durable, allowing for washing.
• Quantitative and qualitative experiments validated the framework's performance.

Conclusions:

• The VIST framework offers a significant advancement in hand tracking technology.
• This approach has broad potential applications in human-robot interaction and virtual/augmented reality.
• The system provides a practical and effective solution for complex hand motion tracking needs.