深度强化学习与明确的上下文表示
IEEE transactions on neural networks and learning systems
|October 31, 2023
概括
本研究介绍了IOTA显式上下文表示 (IECR) 框架,使强化学习 (RL) 代理商能够更有效地从上下文信息中学习. 使用IECR的新算法在离散环境中明显优于现有方法.
科学领域:
- 人工智能的人工智能
- 机器学习 机器学习
- 强化学习是一种强化学习.
背景情况:
- 强化学习 (RL) 在复杂的问题上表现出色,但经常与上下文信息作斗争.
- 人类决策有效地利用上下文来避免错误,这是许多RL代理缺乏的能力.
- 目前的RL代理需要广泛的培训,以避免人类通过上下文识别的明显错误.
研究的目的:
- 提出一个新的框架,IOTA显式上下文表示 (IECR),用于离散环境.
- 提高RL代理人从上下文信息中学习的能力.
- 通过结合明确的背景来提高学习效率和绩效.
主要方法:
- 使用上下文关键框架 (CKFs) 来表示状态.
- 从CKF中提取国家支出的函数.
- 引入两个与国家支付相关的新浪损失函数.
- 开发了四个新的上下文感知RL算法:IDQN,IDDQN,IDuDQN,IDDDQN.
主要成果:
- IECR框架成功地从上下文信息中提取和学习.
- 使用IECR开发的所有算法都显著提高了性能.
- 算法在大约4万个神经网络训练步骤内融合,超过了最先进的同等效果.
结论:
- IECR框架提供了一种有效的方法,用于将明确的上下文纳入RL.
- 语境学习显著加快了融合,并提高了在离散环境中的性能.
- 拟议的算法为RL代理提供了实质性的进步,他们运行时具有上下文意识.
相关概念视频
Reinforcement
221
Positive and negative reinforcement are key concepts in operant conditioning, a learning process where the consequences of a behavior affect the likelihood of that behavior being repeated.
Positive reinforcement occurs when a behavior is followed by the presentation of a rewarding stimulus, increasing the frequency of that behavior. For example:
Positive reinforcement occurs when a behavior is followed by the presentation of a rewarding stimulus, increasing the frequency of that behavior. For example:
221
Associative Learning
412
Associative learning is a fundamental concept in behavioral psychology, wherein a connection is established between two stimuli or events, leading to a learned response. This process is critical in understanding how behaviors are acquired and modified. Conditioning, the mechanism through which associations are formed, can be divided into two main types: classical conditioning and operant conditioning, each elucidating different aspects of associative learning.
Classical conditioning, also known...
Classical conditioning, also known...
412
Elaborative Rehearsals
90
Elaborative rehearsal is a crucial cognitive strategy that strengthens information encoding in long-term memory by making meaningful connections between new data and pre-existing knowledge. This approach contrasts with maintenance rehearsal, which involves simple repetition without delving into the significance of the information. While maintenance rehearsal might temporarily keep information active in short-term memory, it is less effective for long-term retention.
The effectiveness of...
The effectiveness of...
90
Implicit Memories
135
Implicit memories, also known as non-declarative memories, are long-term memories that function outside of conscious awareness. These memories influence behavior and skills without explicit knowledge. This type of memory is evident in tasks like playing tennis, snowboarding, and texting. Implicit memory has three subsystems: procedural memory, conditioning, and priming. This type of memory is essential in various activities, from everyday tasks to specialized skills.
One key aspect of implicit...
One key aspect of implicit...
135
Observational Learning
188
Albert Bandura's observational learning, also known as imitation or modeling, occurs when a person observes and imitates another's behavior. It is a quicker process than operant conditioning. A well-known example is the Bobo doll study, where children who saw an adult acting aggressively towards the doll were more likely to act aggressively when left alone, compared to those who observed a nonaggressive adult. Many psychologists view observational learning as a form of latent learning...
188
Purposive Learning
123
E. C. Tolman emphasized the purposiveness of behavior — the idea that much of our behavior is goal-directed. For instance, employees who aim for a promotion work diligently to meet their targets. Tolman argued that when classical conditioning and operant conditioning occur, the organism acquires certain expectations. In classical conditioning, a child might fear a dog because they expect it to bite. In operant conditioning, a person might consistently work overtime because they expect a...
123


