Real-time human progress estimation with online dynamic time warping for collaborative robotics

Area of Science:

• Robotics
• Human-Computer Interaction
• Machine Learning

Background:

• Real-time human action progress estimation is crucial for effective human-robot collaboration.
• Current methods for action progress estimation are underexplored, limiting seamless collaboration.

Purpose of the Study:

• To propose the first real-time application of Open-end Soft-DTW (OS-DTW_EU).
• To introduce OS-DTW_WP, a novel DTW variant for capturing local correlations.
• To develop the Proactive Assistance through action-Completion Estimation (PACE) framework for synchronized robotic assistance.

Main Methods:

• Implemented real-time OS-DTW_EU and introduced OS-DTW_WP with Windowed-Pearson distance.
• Integrated these methods into the PACE framework using reinforcement learning for action completion estimation.
• Conducted experiments on a chair assembly task and user studies with 12 participants.

Main Results:

• OS-DTW_WP demonstrated superiority in capturing local motion patterns.
• OS-DTW_EU proved effective for tasks with consistent absolute positions.
• The PACE framework significantly improved interaction fluency, reduced waiting times, and received positive user feedback.

Conclusions:

• The proposed OS-DTW variants and PACE framework offer a significant advancement in real-time human action progress estimation.
• This research enables more fluid and efficient human-robot collaboration.
• The findings highlight the potential for adaptive robotic assistance in complex tasks.