Personalized Markerless Upper-Body Tracking with a Depth Camera and Wrist-Worn Inertial Measurement Units

Area of Science:

• Biomedical Engineering
• Computer Vision
• Human-Computer Interaction

Background:

• Marker-based motion capture is accurate but cumbersome.
• Markerless systems offer convenience but often lack precision, especially during occlusions.
• Existing depth-camera-based systems like Kinect have limitations in upper-body joint tracking accuracy.

Purpose of the Study:

• To develop an accurate markerless motion capture technique.
• To improve upper-body joint tracking accuracy using sensor fusion.
• To provide a publicly available dataset for research.

Main Methods:

• Fusion of depth camera (Kinect V2) and wrist-worn inertial measurement units (IMUs).
• Creation of a personalized articulated human mesh model from a single depth image.
• Utilizing IMUs for enhanced arm tracking, particularly during occlusions.

Main Results:

• Over 20% reduction in upper-limb joint position errors compared to Kinect's native skeleton tracking.
• Demonstrated improved accuracy through evaluation against a marker-based gold standard system.
• Successful fusion of depth and IMU data for robust motion capture.

Conclusions:

• The proposed markerless motion capture system enhances accuracy through sensor fusion and personalized modeling.
• This technique offers a viable alternative to marker-based systems for specific applications.
• The publicly released dataset facilitates further research in human motion analysis.