Interactive Teleoperation of an Articulated Robotic Arm Using Vision-Based Human Hand Tracking

Area of Science:

• Robotics
• Computer Vision
• Human-Robot Interaction

Background:

• Existing teleoperation systems often require specialized sensors or wearable devices, hindering accessibility and scalability.
• Intuitive human-robot interaction is crucial for advancing robotic manipulation and control.

Purpose of the Study:

• To develop and evaluate a vision-based teleoperation framework for controlling an articulated robotic arm using a standard laptop camera.
• To enable real-time robotic arm control through human hand tracking and gesture recognition.

Main Methods:

• Utilized a real-time landmark estimation pipeline for hand pose and gesture extraction.
• Mapped kinematic descriptors (palm position, scale, wrist rotation, pitch, pinch gesture) to robotic joint commands via a calibration-based strategy.
• Implemented temporal smoothing and rate-limited updates for enhanced stability and responsiveness.

Main Results:

• Achieved an 88% success rate in human-in-the-loop evaluations with 42 participants.
• Demonstrated a mean completion time of 53.48 ± 18.51 seconds and a placement error of 6.73 ± 3.11 cm for successful trials.
• Reported an ease-of-use score of 2.67 ± 1.20 on a 1-5 scale.

Conclusions:

• The proposed vision-based framework enables feasible interactive teleoperation without specialized hardware.
• The system offers a low-cost platform with potential applications in robotic manipulation, education, and rapid prototyping.