A Novel Multi-Modal Teleoperation of a Humanoid Assistive Robot with Real-Time Motion Mimic

Area of Science:

• Robotics
• Human-Computer Interaction
• Virtual Reality

Background:

• Assistive robots require intuitive control methods for diverse applications.
• Teleoperation systems can bridge the gap between operators and robots in various environments.

Purpose of the Study:

• To develop and evaluate a versatile teleoperation system for the NAO assistive robot.
• To enable intuitive control of robot movement, arm, and head articulation.

Main Methods:

• Integration of Kinect V2 sensor, Meta Quest VR glasses, and Nintendo Switch Joycons.
• Utilization of the Robot Operating System (ROS) for device communication.
• Implementation of two interchangeable control modes: exclusive controller for movement and sensor-based control for articulation.

Main Results:

• Successful implementation of a dual-mode teleoperation system for the NAO robot.
• First-person visual feedback and voice interaction capabilities were achieved.
• The system demonstrated promising control for both long-distance travel and fine-motor tasks.

Conclusions:

• The developed teleoperation system offers flexible and intuitive control for assistive robots.
• The system has significant potential for educational and therapeutic applications.
• Future work can further refine control schemes and expand application domains.