基于稀疏编码的动态价值估计网络,用于深度强化学习
Haoli Zhao1, Zhenni Li2, Wensheng Su2
1School of Automation, Guangdong University of Technology, Guangzhou, 510006, China; Guangdong-HongKong-Macao Joint Laboratory for Smart Discrete Manufacturing, Guangzhou 510006, China.
概括
动态 Sparse 编码通过减少干扰和提高效率来增强深度强化学习 (DRL) 值估计网络. 这种方法导致了更好的控制性能和在各种DRL应用中更快的融合.
科学领域:
- 人工智能的人工智能
- 机器学习 机器学习
- 控制系统 控制系统
背景情况:
- 深度强化学习 (DRL) 对于控制自动化至关重要.
- 在DRL中的估值网络 (VEN) 容易受到灾难性的干扰.
- 准确的价值估计是DLR性能的关键.
研究的目的:
- 提出一个基于动态稀疏编码 (DSC) 的VEN模型.
- 提高价值预测的准确性和DLR的培训效率.
- 针对VEN中的干扰和冗余参数.
主要方法:
- 在VEN中实现了用于稀疏表示和动态梯度的DSC.
- 使用动态值,以实现高效的重量修剪.
- 将DSC-VEN模型应用于离散动作 (Q学习) 和连续动作 (演员-批判) DRL.
主要成果:
- 与基准DRL算法相比,实现了更高的控制性能.
- 演示显著的性能提高 (例如,在Puddle World中>25%,在Hopper中~10%).
- 展示了在不同环境中更少的情节的高效融合.
结论:
- 基于DSC的VEN提供精确的稀疏表示,用于准确的价值预测.
- DSC有效地减轻干扰,并削减多余的参数.
- 拟议的方法提高了在各种控制任务中DRL的性能和培训效率.
相关概念视频
Reinforcement
266
Positive and negative reinforcement are key concepts in operant conditioning, a learning process where the consequences of a behavior affect the likelihood of that behavior being repeated.
Positive reinforcement occurs when a behavior is followed by the presentation of a rewarding stimulus, increasing the frequency of that behavior. For example:
Positive reinforcement occurs when a behavior is followed by the presentation of a rewarding stimulus, increasing the frequency of that behavior. For example:
266
Expected Value
4.0K
The expected value is known as the "long-term" average or mean. This means that over the long term of experimenting over and over, you would expect this average. The expected average is represented by the symbol μ. It is calculated as follows:
4.0K
Reinforcement Schedules
197
Positive reinforcement is a powerful method for teaching new behaviors to both animals and humans. B.F. Skinner demonstrated this with his experiments using rats in a Skinner box. When a rat pressed a lever, it received a food pellet. This immediate reward encouraged the rat to repeat the behavior. This method, where a reward follows every instance of the behavior, is known as continuous reinforcement. It is highly effective for establishing new behaviors quickly.
Once a behavior is learned,...
Once a behavior is learned,...
197
Observational Learning
202
Albert Bandura's observational learning, also known as imitation or modeling, occurs when a person observes and imitates another's behavior. It is a quicker process than operant conditioning. A well-known example is the Bobo doll study, where children who saw an adult acting aggressively towards the doll were more likely to act aggressively when left alone, compared to those who observed a nonaggressive adult. Many psychologists view observational learning as a form of latent learning...
202
Decision Making: P-value Method
5.5K
The process of hypothesis testing based on the P-value method includes calculating the P- value using the sample data and interpreting it.
First, a specific claim about the population parameter is proposed. The claim is based on the research question and is stated in a simple form. Further, an opposing statement to the claim is also stated. These statements can act as null and alternative hypotheses: a null hypothesis would be a neutral statement while the alternative hypothesis can...
First, a specific claim about the population parameter is proposed. The claim is based on the research question and is stated in a simple form. Further, an opposing statement to the claim is also stated. These statements can act as null and alternative hypotheses: a null hypothesis would be a neutral statement while the alternative hypothesis can...
5.5K
One-Compartment Open Model: Wagner-Nelson and Loo Riegelman Method for ka Estimation
556
This lesson introduces two critical methods in pharmacokinetics, the Wagner-Nelson and Loo-Riegelman methods, used for estimating the absorption rate constant (ka) for drugs administered via non-intravenous routes. The Wagner-Nelson method relates ka to the plasma concentration derived from the slope of a semilog percent unabsorbed time plot. However, it is limited to drugs with one-compartment kinetics and can be impacted by factors like gastrointestinal motility or enzymatic degradation.
On...
On...
556


