Experimental Study of a Deep-Learning RGB-D Tracker for Virtual Remote Human Model Reconstruction

Area of Science:

• Computer Vision
• Human-Computer Interaction
• Biomedical Engineering

Background:

• Tracking human body movements in natural environments is crucial for applications like anomaly detection and remote monitoring.
• Virtual avatar models can visualize and analyze tracked movement data.
• Deep learning and RGB-D sensors offer potential for real-time human tracking and reconstruction.

Purpose of the Study:

• To experimentally evaluate a commercial deep-learning body tracking system using an RGB-D sensor for virtual human model reconstruction.
• To assess the system's robustness and identify limitations under natural indoor conditions.
• To develop and investigate a noise compensation method for improving skeleton data quality.

Main Methods:

• Utilized a commercially available deep-learning body tracking system with an RGB-D sensor.
• Conducted experiments in an indoor environment under natural living conditions.
• Analyzed skeleton data (joint positions) to evaluate tracker performance and noise.
• Developed and applied a novel approach to compensate for noise in joint coordinate data.

Main Results:

• The deep-learning tracking system is susceptible to environmental factors, introducing noise in skeleton joint estimations.
• This noise presents challenges for accurate virtual human model reconstruction.
• The proposed noise compensation method demonstrated improved temporal variation of joint coordinates.
• Extracted joint position data proved valuable for virtual human model reconstruction.

Conclusions:

• Deep-learning based RGB-D body tracking shows promise but requires robust handling of environmental noise.
• The developed noise compensation technique enhances the reliability of skeleton data for virtual human modeling.
• Further research can leverage this improved data for advanced applications in human movement analysis and monitoring.