积极推断和强化学习:在部分可观测性下对连续状态和动作空间的统一推断
Parvin Malekzadeh1, Konstantinos N Plataniotis2
1Edward S. Rogers Sr. Department of Electrical and Computer Engineering, University of Toronto, M5S 3G8, Canada p.malekzadeh@mail.utoronto.ca.
Neural computation
|August 23, 2024
概括
这项研究将强化学习 (RL) 和主动推理 (AIF) 统一起来,为部分可观测的环境创建更好的决策代理. 新方法改善了连续空间的学习,并使奖励设计成为可选的.
科学领域:
- 人工智能的人工智能
- 计算神经科学是一种神经科学.
背景情况:
- 强化学习 (RL) 在完全可观测的环境中表现出色,但在现实场景中常见的部分观测中扎.
- 部分可观测的马尔科夫决策过程 (POMDP) 模拟这些复杂的环境,但传统的RL方法面临着长时间视界和高维数据的挑战.
- 积极推理 (AIF) 通过将预期的自由能量 (EFE) 降至最低,平衡勘探和开采,提供了一个替代方案,但在大型空间中受到计算需求的限制.
研究的目的:
- 提出一个统一的原则,将RL和AIF连接起来,以便在POMDP中加强代理人的决策.
- 克服现有的RL和AIF方法在连续,高维和长视界部分可观测环境中的局限性.
- 展示一种新的方法,它结合了奖励最大化和信息搜索行为.
主要方法:
- 开发了一个统一RL和AIF原则的理论框架.
- 制定了一种适用于连续空间POMDPs的新方法.
- 进行了严格的理论分析和实验验证.
主要成果:
- 在AIF和RL之间建立了理论上的联系,使无集成成为可能.
- 与现有的RL方法相比,在连续空间POMDP中表现出优越的学习能力.
- 展示了通过利用信息搜索探索来解决无奖励问题的能力.
结论:
- 统一原则有效地弥合了RL和AIF,克服了个别方法的局限性.
- 拟议的方法提供了一个强大的新工具,用于设计能够处理复杂,部分可观测的环境的人工制剂.
- 这项工作为人工智能领域的AIF开辟了新的途径,减少了对明确奖励规范的依赖.
相关概念视频
Observational Learning
154
Albert Bandura's observational learning, also known as imitation or modeling, occurs when a person observes and imitates another's behavior. It is a quicker process than operant conditioning. A well-known example is the Bobo doll study, where children who saw an adult acting aggressively towards the doll were more likely to act aggressively when left alone, compared to those who observed a nonaggressive adult. Many psychologists view observational learning as a form of latent learning...
154
State Space Representation
178
The frequency-domain technique, commonly used in analyzing and designing feedback control systems, is effective for linear, time-invariant systems. However, it falls short when dealing with nonlinear, time-varying, and multiple-input multiple-output systems. The time-domain or state-space approach addresses these limitations by utilizing state variables to construct simultaneous, first-order differential equations, known as state equations, for an nth-order system.
Consider an RLC circuit, a...
Consider an RLC circuit, a...
178
Reinforcement
192
Positive and negative reinforcement are key concepts in operant conditioning, a learning process where the consequences of a behavior affect the likelihood of that behavior being repeated.
Positive reinforcement occurs when a behavior is followed by the presentation of a rewarding stimulus, increasing the frequency of that behavior. For example:
Positive reinforcement occurs when a behavior is followed by the presentation of a rewarding stimulus, increasing the frequency of that behavior. For example:
192
Associative Learning
318
Associative learning is a fundamental concept in behavioral psychology, wherein a connection is established between two stimuli or events, leading to a learned response. This process is critical in understanding how behaviors are acquired and modified. Conditioning, the mechanism through which associations are formed, can be divided into two main types: classical conditioning and operant conditioning, each elucidating different aspects of associative learning.
Classical conditioning, also known...
Classical conditioning, also known...
318
Reinforcement Schedules
138
Positive reinforcement is a powerful method for teaching new behaviors to both animals and humans. B.F. Skinner demonstrated this with his experiments using rats in a Skinner box. When a rat pressed a lever, it received a food pellet. This immediate reward encouraged the rat to repeat the behavior. This method, where a reward follows every instance of the behavior, is known as continuous reinforcement. It is highly effective for establishing new behaviors quickly.
Once a behavior is learned,...
Once a behavior is learned,...
138
Avoidance Learning and Learned Helplessness
1.7K
Avoidance learning and learned helplessness are critical concepts in understanding behavioral responses to negative stimuli.
Avoidance learning occurs when an organism learns that a specific behavior can prevent an unpleasant outcome. For example, a student who receives a bad grade may start studying harder to avoid future poor grades. This behavior persists even when the negative outcome is no longer present. Avoidance learning is powerful because it maintains behavior in the absence of the...
Avoidance learning occurs when an organism learns that a specific behavior can prevent an unpleasant outcome. For example, a student who receives a bad grade may start studying harder to avoid future poor grades. This behavior persists even when the negative outcome is no longer present. Avoidance learning is powerful because it maintains behavior in the absence of the...
1.7K


