研究基于深度学习的人机器人静态/动态手势驱动控制框架的研究
Gong Zhang1,2, Jiahong Su2,3, Shuzhong Zhang3
1School of Automation, Guangdong Polytechnic Normal University, Guangzhou 510665, China.
Sensors (Basel, Switzerland)
|December 11, 2025
概括
本研究介绍了一种深度学习方法,用于使用静态和动态手势控制机器人. 该方法在手势识别和成功完成对象操纵任务方面实现了高精度,增强了人机协作.
科学领域:
- 机器人技术 机器人技术 机器人技术
- 人工智能的人工智能
- 计算机视觉 计算机视觉
背景情况:
- 人机交互 (HRI) 对于协作任务至关重要.
- 手势驱动的控制为机器人提供了一个自然的界面.
- 在不同的条件下强度对于实际的HRI至关重要.
研究的目的:
- 开发一种基于深度学习的系统,用于以手势驱动的机器人控制.
- 为了使机器人能够使用静态和动态的手势来执行物体抓取和交付任务.
- 在各种照明条件下评估系统的性能和稳定性.
主要方法:
- 利用2D卷积神经网络 (2D-CNNs) 进行静态手势识别.
- 采用混合3D卷积神经网络 (3D-CNNs) 和长短期记忆 (3D-CNN+LSTM) 网络进行动态手势识别.
- 集成的MediaPipe用于手部特征提取和深度摄像头用于3D姿势估计.
主要成果:
- 实现了对静态手势的95.38%和动态手势的93.18%的验证准确度.
- 在100个试验中,每个参与者的平均任务成功率不低于96.88% (静态) 和94.63% (动态).
- 在自然,低光和强光条件下,始终保持任务完成时间在20秒内.
结论:
- 拟议的深度学习方法可以使用自然手势进行基于视觉的机器人控制.
- 该系统有效地促进了对象抓取和交付任务,显示了高精度和可靠性.
- 这项研究对推进人机协作应用具有重大前景.
相关概念视频
01:18Hierarchy of Motor Control
5.8K
The hierarchy of motor control refers to the different levels of organization and processing involved in controlling movement in the body. These levels range from higher cortical areas involved in planning and decision-making to lower spinal cord reflexes that respond automatically to external stimuli.
5.8K
01:17Open and closed-loop control systems
1.5K
Control systems are foundational elements in automation and engineering. They are broadly categorized into open-loop and closed-loop systems. These classifications hinge on the presence or absence of feedback mechanisms, significantly influencing the system's performance, complexity, and application.
An open-loop control system operates without feedback from the output. It consists of two primary elements: the controller and the controlled process. The controller receives an input signal...
An open-loop control system operates without feedback from the output. It consists of two primary elements: the controller and the controlled process. The controller receives an input signal...
1.5K
01:24One-Degree-of-Freedom System
767
In mechanical engineering, one-degree-of-freedom systems form the basis of a wide range of electrical and mechanical components. Using these models, engineers can predict the behavior of various parts in a larger system, which gives them insight into how different forces interact with each other.
A one-degree-of-freedom system is defined by an independent variable that determines its state and behavior. One example of a one-degree-of-freedom system is a simple harmonic oscillator, such as a...
A one-degree-of-freedom system is defined by an independent variable that determines its state and behavior. One example of a one-degree-of-freedom system is a simple harmonic oscillator, such as a...
767
01:26PD Controller: Design
588
In automotive engineering, car suspension systems often employ Proportional Derivative (PD) controllers to enhance performance. PD controllers are utilized to adjust the damping force in response to road conditions. A controller, acting as an amplifier with a constant gain, demonstrates proportional control, with output directly mirroring input.
Designing a continuous-data controller requires selecting and linking components like adders and integrators, which are fundamental in Proportional,...
Designing a continuous-data controller requires selecting and linking components like adders and integrators, which are fundamental in Proportional,...
588