Motion Estimation and Hand Gesture Recognition-Based Human-UAV Interaction Approach in Real Time.

Area of Science:

• Robotics
• Human-Computer Interaction
• Computer Vision

Background:

• Traditional remote controllers and vision-based gesture systems face limitations in real-time performance and accuracy for dynamic hand gestures in human-UAV interaction (HUI).
• Estimating multi-dimensional hand poses from 2D images is challenging, leading to complex algorithms unsuitable for safety-critical HUI systems.

Purpose of the Study:

• To propose a hybrid hand gesture system integrating an inertial measurement unit (IMU)-based motion capture system with a vision-based gesture system.
• To enhance the real-time performance and intuitiveness of human-UAV interaction (HUI) systems.

Main Methods:

• A hybrid system combining IMU-based and vision-based gesture recognition was developed.
• IMU-based commands utilized thumb-mounted micro-IMU orientation for intuitive UAV directional control.
• Vision-based commands employed real-time object detection for hand appearance mapping.

Main Results:

• The hybrid system demonstrated a processing speed of 0.089 ms.
• Usability testing showed an average lap time approximately 19 seconds faster than traditional joystick controllers.
• Efficiency was evaluated against existing vision-based systems for dynamic gestures.

Conclusions:

• The proposed hybrid system offers a safer and more intuitive HUI design.
• The system is viable as an alternative to existing HUI methods, significantly improving real-time performance.
• This approach addresses the limitations of purely vision-based or traditional control systems for UAV operation.