Related Experiment Video
Updated: Dec 17, 2025
11:53
The Modular Design and Production of an Intelligent Robot Based on a Closed-Loop Control Strategy
Published on: October 14, 2017
12.1K
A Multitasking-Oriented Robot Arm Motion Planning Scheme Based on Deep Reinforcement Learning and Twin
Chuzhao Liu1,2, Junyao Gao1,2, Yuanzhen Bi1,2
1Intelligent Robotics Institute, School of Mechatronical Engineering, Beijing Institute of Technology, 5 Nandajie, Zhongguancun, Haidian, Beijing 100081, China.
Sensors (Basel, Switzerland)
|June 25, 2020
Summary
This study introduces a novel deep reinforcement learning (DRL) and digital twin approach for controlling humanoid robot arms. The method enables rapid, stable, and diverse multitasking for robots like the BHR-6, improving learning efficiency.
Area of Science:
- Robotics and Artificial Intelligence
- Humanoid Robot Control Systems
- Digital Twin Technology Applications
Background:
- Humanoid robots with arms are crucial for public acceptance and present significant robotics challenges.
- Digital twin technology aligns with Industry 4.0 and Made in China 2025 initiatives.
- Existing methods struggle with rapid, diverse, and stable motion planning for humanoid robot arms.
Purpose of the Study:
- To propose a combined deep reinforcement learning (DRL) and digital twin scheme for controlling humanoid robot arms.
- To develop a multitasking-oriented training approach for rapid and stable motion planning.
- To enhance DRL training efficiency by incorporating a priori knowledge and improving reward functions.
Main Methods:
- A Twin Synchro-Control (TSC) scheme integrating DRL with digital twin technology for robot arm control.
- Development of a data acquisition system to generate human joint angle data for training.
- Utilizing human joint angle data to refine the reward function of the Deep Deterministic Policy Gradient (DDPG) algorithm.
- Application to the BHR-6 humanoid robot model for simulation-based training.
Main Results:
- The proposed DRL with TSC scheme enables fast and diverse multitasking for humanoid robot arms.
- The approach effectively addresses the sparse reward problem in DRL using collected human motion data.
- Simulations demonstrated superior learning stability and convergence speed compared to vanilla DDPG.
- The trained humanoid robot successfully performed tasks not achievable with existing training methods.
Conclusions:
- The integration of DRL and digital twin technology offers a powerful solution for humanoid robot arm control.
- The TSC scheme with a priori knowledge significantly improves training efficiency and task performance.
- This method facilitates rapid adaptation and multi-task capability in complex humanoid robots.
Related Concept Videos
01:22Planar Rigid-Body Motion
862
Understanding the movement of a rigid body in planar motion involves recognizing that every particle within this body is traversing a path that maintains a consistent distance from a specific plane. This concept is fundamental in the study of physics and mechanical engineering, and it allows us to comprehend better how objects move in space.
Planar motion is typically divided into three distinct categories. The first is rectilinear translation, demonstrated by a subway train that moves along...
Planar motion is typically divided into three distinct categories. The first is rectilinear translation, demonstrated by a subway train that moves along...
862
01:24One-Degree-of-Freedom System
710
In mechanical engineering, one-degree-of-freedom systems form the basis of a wide range of electrical and mechanical components. Using these models, engineers can predict the behavior of various parts in a larger system, which gives them insight into how different forces interact with each other.
A one-degree-of-freedom system is defined by an independent variable that determines its state and behavior. One example of a one-degree-of-freedom system is a simple harmonic oscillator, such as a...
A one-degree-of-freedom system is defined by an independent variable that determines its state and behavior. One example of a one-degree-of-freedom system is a simple harmonic oscillator, such as a...
710
01:18Hierarchy of Motor Control
5.7K
The hierarchy of motor control refers to the different levels of organization and processing involved in controlling movement in the body. These levels range from higher cortical areas involved in planning and decision-making to lower spinal cord reflexes that respond automatically to external stimuli.
5.7K
01:24Muscle Coordination and Action
2.9K
Muscle coordination is a complex and finely tuned process essential for smooth and purposeful movements like flexion, extension, adduction, abduction, and rotation. The human body orchestrates the actions of various muscles working in concert, each with a specific role. Four functional types describe how muscles work together: agonist, antagonist, synergist, and fixator.
Agonists
Agonist muscles, often called prime movers, are the primary muscles responsible for producing a specific movement....
Agonists
Agonist muscles, often called prime movers, are the primary muscles responsible for producing a specific movement....
2.9K
01:30Three-Dimensional Force System:Problem Solving
1.3K
A three-dimensional force system refers to a scenario in which three forces act simultaneously in three different directions. This type of problem is commonly encountered in physics and engineering, where it is necessary to calculate the resultant force on the system, which can then be used to predict or analyze the behavior of the object or structure under consideration.
To solve a three-dimensional force system, first resolve each force into its respective scalar components. Do this using...
To solve a three-dimensional force system, first resolve each force into its respective scalar components. Do this using...
1.3K
01:17Open and closed-loop control systems
1.4K
Control systems are foundational elements in automation and engineering. They are broadly categorized into open-loop and closed-loop systems. These classifications hinge on the presence or absence of feedback mechanisms, significantly influencing the system's performance, complexity, and application.
An open-loop control system operates without feedback from the output. It consists of two primary elements: the controller and the controlled process. The controller receives an input signal...
An open-loop control system operates without feedback from the output. It consists of two primary elements: the controller and the controlled process. The controller receives an input signal...
1.4K