HG2P:受海马体启发的高奖励图表和无模型的Q梯度惩罚,用于路径规划和运动控制
Haoran Wang1, Yaoru Sun1, Zeshen Tang1
1Department of Computer Science and Technology, Tongji University, Shanghai, 201804, China.
概括
这项研究介绍了HG2P+ACLG,这是一个由大脑机制启发的新型目标条件下等级强化学习 (HRL) 框架. 它提高了复杂的长视野任务的样本效率和概括性,例如导航和机器人操纵.
科学领域:
- 人工智能的人工智能
- 神经科学是一个神经科学.
- 机器人技术 机器人技术 机器人技术
背景情况:
- 以目标为条件的层次增强学习 (HRL) 有效地将复杂的任务分解为可管理的子目标.
- 现有的HRL方法在长期规划和在大规模环境中的泛化方面面临挑战.
- 将HRL与大脑机制相结合,有可能改善学习策略.
研究的目的:
- 提出一个新的目标条件的HRL框架,HG2P+ACLG,将基于图表的规划与大脑启发的机制集成在一起.
- 通过结合高回报采样和无模型梯度处罚来提高HRL中的采样效率和概括性.
- 验证关于复杂的远程导航和机器人操纵任务的拟议框架.
主要方法:
- 开发了一种类似于海马-状体的双控制器假设,用于目标条件的HRL.
- 介绍了一种高回报率采样策略用于内存图形构建,灵感来自海马重复和基于实例的理论.
- 导出了一个无模型的低级Q函数梯度惩罚,以解决模型依赖性和改进Lipschitz约束概括.
- 将这些组件集成到ACLG框架中,创建HG2P+ACLG.
主要成果:
- 与最先进的目标条件HRL算法相比,HG2P+ACLG框架显示出更高的性能.
- 在各种长视界导航任务中观察到显著的改进.
- 在复杂的机器人操纵任务中,性能提升也很明显.
结论:
- 拟议的HG2P+ACLG框架为目标条件的HRL提供了有前途的进展.
- 脑启发机制和新型算法组件的整合导致更好的样本效率和概括性.
- 这种方法有效地解决了复杂的现实应用的长期规划方面的挑战.
相关概念视频
01:18Hierarchy of Motor Control
3.7K
The hierarchy of motor control refers to the different levels of organization and processing involved in controlling movement in the body. These levels range from higher cortical areas involved in planning and decision-making to lower spinal cord reflexes that respond automatically to external stimuli.
3.7K
01:22Planar Rigid-Body Motion
561
Understanding the movement of a rigid body in planar motion involves recognizing that every particle within this body is traversing a path that maintains a consistent distance from a specific plane. This concept is fundamental in the study of physics and mechanical engineering, and it allows us to comprehend better how objects move in space.
Planar motion is typically divided into three distinct categories. The first is rectilinear translation, demonstrated by a subway train that moves along...
Planar motion is typically divided into three distinct categories. The first is rectilinear translation, demonstrated by a subway train that moves along...
561
01:16Hydraulic Jump: Problem Solving
148
To analyze a hydraulic jump in a rectangular channel with a flow speed of 6 meters per second, follow these steps:Calculate Effective Upstream Velocity:When the downstream gate closes, a hydraulic jump forms, traveling upstream at 2 meters per second. This wave speed combines with the initial channel flow velocity, creating an effective upstream velocity.Identify Flow Velocities Before and After the Hydraulic Jump:Upstream of the hydraulic jump, the effective flow velocity includes both the...
148
01:29Relative Motion Analysis using Rotating Axes-Problem Solving
453
Consider a crane whose telescopic boom rotates with an angular velocity of 0.04 rad/s and angular acceleration of 0.02 rad/s2. Along with the rotation, the boom also extends linearly with a uniform speed of 5 m/s. The extension of the boom is measured at point D, which is measured with respect to the fixed point C on the other end of the boom. For the given instant, the distance between points C and D is 60 meters.
Here, in order to determine the magnitude of velocity and acceleration for point...
Here, in order to determine the magnitude of velocity and acceleration for point...
453
01:30Three-Dimensional Force System:Problem Solving
866
A three-dimensional force system refers to a scenario in which three forces act simultaneously in three different directions. This type of problem is commonly encountered in physics and engineering, where it is necessary to calculate the resultant force on the system, which can then be used to predict or analyze the behavior of the object or structure under consideration.
To solve a three-dimensional force system, first resolve each force into its respective scalar components. Do this using...
To solve a three-dimensional force system, first resolve each force into its respective scalar components. Do this using...
866
01:22Indirect Motor Pathways
1.7K
The indirect motor or extrapyramidal pathways originate in the brainstem, the lower portion of the brain that connects it to the spinal cord. They consist of several distinct tracts, each with specialized functions. The four main tracts of the indirect motor pathways are the vestibulospinal tract, the reticulospinal tract, the tectospinal tract, and the rubrospinal tract.
The vestibulospinal tract originates in the vestibular nuclei of the brainstem. The vestibular system detects changes in...
The vestibulospinal tract originates in the vestibular nuclei of the brainstem. The vestibular system detects changes in...
1.7K