Development of a Visual Perception System on a Dual-Arm Mobile Robot for Human-Robot Interaction

Area of Science:

• Robotics
• Computer Vision
• Human-Robot Interaction

Background:

• Human-robot interaction (HRI) requires intuitive communication and understanding.
• Mobile robots need advanced perception systems for effective collaboration with humans.

Purpose of the Study:

• To develop and validate a visual-perception system for a dual-arm mobile robot.
• To enhance human-robot interaction through integrated perception subsystems.

Main Methods:

• Integration of three subsystems: hand gesture recognition, cognitive detection of engagement/intention, and visual servoing with YOLO object detection and model-based tracking.
• Implementation on a dual-arm mobile robot platform.

Main Results:

• The visual-perception system successfully integrates gesture recognition, cognitive state detection, and object tracking.
• Experimental validation demonstrates the system's effectiveness in human-robot interaction applications.

Conclusions:

• The developed visual-perception system enhances the capabilities of dual-arm mobile robots in HRI scenarios.
• The integrated approach provides a robust framework for intuitive and effective human-robot collaboration.