Human intention recognition by deep LSTM and transformer networks for real-time human-robot collaboration

Area of Science:

• Robotics
• Artificial Intelligence
• Human-Computer Interaction

Background:

• Human-robot collaboration (HRC) is crucial for industrial efficiency, safety, and reducing worker strain.
• Real-time intention recognition is key for seamless task sharing and interaction in HRC systems.
• Existing methods often lack robustness in predicting worker intentions for dynamic environments.

Purpose of the Study:

• To develop and validate an integrated HRC system with advanced intention recognition.
• To improve real-time task sharing, robot motion control, and safety in industrial settings.
• To offer a more intuitive and efficient collaborative experience between humans and robots.

Main Methods:

• Utilized state-of-the-art human pose estimation and deep learning models.
• Employed LSTM-based and transformer-based neural networks for hand trajectory classification.
• Integrated dynamic movement primitives (DMPs) for robot motion control and safety features.

Main Results:

• Achieved higher accuracy in classifying human hand trajectories compared to previous methods.
• Demonstrated effective real-time intention detection and prediction.
• Validated enhanced fluency, safety, and efficiency in a real-world industrial assembly task.

Conclusions:

• The proposed HRC system significantly improves real-time decision-making in collaborative environments.
• The integration of advanced intention recognition and DMPs offers a safer and more intuitive human-robot interaction.
• This approach shows strong potential for advancing industrial HRC applications.