广泛的批判性深度演员强化学习,用于持续控制
概括
本研究介绍了一种混合深度强化学习 (DRL) 框架,将广泛的学习系统 (BLS) 和深度神经网络 (DNN) 结合起来. 经过BLS改进的DRL算法显示了对持续控制任务的提高训练效率和准确性.
科学领域:
- 人工智能的人工智能
- 机器学习 机器学习
- 机器人技术 机器人技术 机器人技术
背景情况:
- 深度强化学习 (DRL) 在持续控制方面表现有前途,但由于深度神经网络 (DNN),需要大量的数据和计算.
- 现有的DRL方法面临训练效率和计算成本的挑战,限制了实时应用.
研究的目的:
- 提出一种新的混合行为者-关键强化学习 (RL) 框架,将广泛学习系统 (BLS) 与DNNs集成在一起.
- 为了提高DRL算法的效率和准确性,用于连续控制任务.
主要方法:
- 开发了一个混合框架,批评网络使用BLS进行快速值估计 (回归),演员网络使用DNN进行政策梯度优化.
- 将BLS-DNN混合方法集成到深度决定性政策梯度 (DDPG),软行为者-关键 (SAC) 和双延迟 DDPG (TD3) 算法中,创建了BLS增强的变体.
主要成果:
- 所有通过BLS增强的演员-关键算法都显示出与原始同行相比更高的训练效率.
- 增加了BLS的DLR变种在连续控制任务中实现了更高的准确性.
- 实验结果证实了混合BLS-DNN方法的有效性.
结论:
- 拟议的混合BLS-DNN框架显著提高了DRL训练的效率和准确性.
- 这种方法适用于需要计算效率和快速适应的实时控制场景.
- 可通用的混合设计增强了现有的演员关键RL算法.
相关概念视频
01:23Reinforcement
161
Positive and negative reinforcement are key concepts in operant conditioning, a learning process where the consequences of a behavior affect the likelihood of that behavior being repeated.
Positive reinforcement occurs when a behavior is followed by the presentation of a rewarding stimulus, increasing the frequency of that behavior. For example:
Positive reinforcement occurs when a behavior is followed by the presentation of a rewarding stimulus, increasing the frequency of that behavior. For example:
161
01:12Observational Learning
106
Albert Bandura's observational learning, also known as imitation or modeling, occurs when a person observes and imitates another's behavior. It is a quicker process than operant conditioning. A well-known example is the Bobo doll study, where children who saw an adult acting aggressively towards the doll were more likely to act aggressively when left alone, compared to those who observed a nonaggressive adult. Many psychologists view observational learning as a form of latent learning...
106
01:24Reinforcement Schedules
119
Positive reinforcement is a powerful method for teaching new behaviors to both animals and humans. B.F. Skinner demonstrated this with his experiments using rats in a Skinner box. When a rat pressed a lever, it received a food pellet. This immediate reward encouraged the rat to repeat the behavior. This method, where a reward follows every instance of the behavior, is known as continuous reinforcement. It is highly effective for establishing new behaviors quickly.
Once a behavior is learned,...
Once a behavior is learned,...
119
01:26Feedback control systems
256
Feedback control systems are categorized in various ways based on their design, analysis, and signal types.
Linear feedback systems are theoretical models that simplify analysis and design. These systems operate under the principle that their output is directly proportional to their input within certain ranges. For instance, an amplifier in a control system behaves linearly as long as the input signal remains within a specific range. However, most physical systems exhibit inherent nonlinearity...
Linear feedback systems are theoretical models that simplify analysis and design. These systems operate under the principle that their output is directly proportional to their input within certain ranges. For instance, an amplifier in a control system behaves linearly as long as the input signal remains within a specific range. However, most physical systems exhibit inherent nonlinearity...
256
01:17Open and closed-loop control systems
577
Control systems are foundational elements in automation and engineering. They are broadly categorized into open-loop and closed-loop systems. These classifications hinge on the presence or absence of feedback mechanisms, significantly influencing the system's performance, complexity, and application.
An open-loop control system operates without feedback from the output. It consists of two primary elements: the controller and the controlled process. The controller receives an input signal...
An open-loop control system operates without feedback from the output. It consists of two primary elements: the controller and the controlled process. The controller receives an input signal...
577
01:22Multi-input and Multi-variable systems
90
Cruise control systems in cars are designed as multi-input systems to maintain a driver's desired speed while compensating for external disturbances such as changes in terrain. The block diagram for a cruise control system typically includes two main inputs: the desired speed set by the driver and any external disturbances, such as the incline of the road. By adjusting the engine throttle, the system maintains the vehicle's speed as close to the desired value as possible.
In the absence...
In the absence...
90