快速值跟踪用于深度强化学习的学习
1Department of Statistics, Purdue University, West Lafayette, IN 47907, USA.
概括
这项研究介绍了Langevinized Kalman Temporal-Difference (LKTD),一种新的强化学习 (RL) 算法. 通过利用卡尔曼过和随机梯度马尔科夫链蒙特卡洛方法,LKTD量化了深度强化学习中的不确定性.
科学领域:
- 人工智能的人工智能
- 机器学习 机器学习
- 控制理论 控制理论
背景情况:
- 强化学习 (RL) 代理人与环境互动,以进行连续的决策.
- 当前的RL算法经常忽视环境随机性和不确定性量化.
- 静态模型专注于点估计,忽视动态相互作用.
研究的目的:
- 介绍一个新的,可扩展的采样算法,用于深度强化学习.
- 解决现有的RL方法在不确定性量化方面的局限性.
- 开发一种方法来量化和监测RL培训期间的不确定性.
主要方法:
- 利用卡尔曼的过模式.
- 介绍Langevin化卡尔曼时间差异 (LKTD) 算法.
- 使用随机梯度马尔科夫链蒙特卡罗 (SGMCMC) 来进行神经网络参数的后置采样.
主要成果:
- 在温和条件下证明LKTD后部样本的趋同到静止分布.
- 能够量化价值函数和模型参数中的不确定性.
- 允许在深度强化学习的政策更新期间监控不确定性.
结论:
- LKTD算法为RL的不确定性量化提供了一个强大的方法.
- LKTD促进了更具适应性和可靠性的强化学习系统.
- 这种方法增强了对代理-环境相互作用的不确定性的理解和管理.
相关概念视频
Reinforcement
992
Positive and negative reinforcement are key concepts in operant conditioning, a learning process where the consequences of a behavior affect the likelihood of that behavior being repeated.
Positive reinforcement occurs when a behavior is followed by the presentation of a rewarding stimulus, increasing the frequency of that behavior. For example:
Positive reinforcement occurs when a behavior is followed by the presentation of a rewarding stimulus, increasing the frequency of that behavior. For example:
992
Reinforcement Schedules
559
Positive reinforcement is a powerful method for teaching new behaviors to both animals and humans. B.F. Skinner demonstrated this with his experiments using rats in a Skinner box. When a rat pressed a lever, it received a food pellet. This immediate reward encouraged the rat to repeat the behavior. This method, where a reward follows every instance of the behavior, is known as continuous reinforcement. It is highly effective for establishing new behaviors quickly.
Once a behavior is learned,...
Once a behavior is learned,...
559
Velocity and Position by Integral Method
8.7K
If acceleration as a function of time is known, then velocity and position functions can be derived using integral calculus. For constant acceleration, the integral equations refer to the first and second kinematic equations for velocity and position functions, respectively.
Consider an example to calculate the velocity and position from the acceleration function. A motorboat is traveling at a constant velocity of 5.0 m/s when it starts to decelerate to arrive at the dock. Its acceleration is...
Consider an example to calculate the velocity and position from the acceleration function. A motorboat is traveling at a constant velocity of 5.0 m/s when it starts to decelerate to arrive at the dock. Its acceleration is...
8.7K
Observational Learning
1.1K
Albert Bandura's observational learning, also known as imitation or modeling, occurs when a person observes and imitates another's behavior. It is a quicker process than operant conditioning. A well-known example is the Bobo doll study, where children who saw an adult acting aggressively towards the doll were more likely to act aggressively when left alone, compared to those who observed a nonaggressive adult. Many psychologists view observational learning as a form of latent learning...
1.1K
Average and Instantaneous Velocity Vectors
8.9K
To calculate other physical quantities in kinematics, the time variable must be introduced. The time variable not only allows us to state where an object is (its position) during its motion, but also how fast it’s moving. The speed at which an object is moving is given by the rate at which the position changes with time. For each position, a particular time is assigned. If the details of the motion at each instant are not important, the rate is usually expressed as the average velocity v.
8.9K
Instantaneous Velocity - I
30.4K
The average velocity during a time interval cannot tell us how fast or in what direction a particle is moving at any given time during the interval. To calculate this, it is important to know the instantaneous velocity, which is the velocity at a specific instant of time or at a specific point along the path. Instantaneous velocity is the quantity that measures how fast an object is moving along its path. In other words, the instantaneous velocity vx of an object is the limit of the average...
30.4K


