在组合路由中强化学习的信息理论内在动机
Ruozhang Xi1, Yao Ni2, Wangyu Wu3
1Krieger School of Arts and Sciences, Johns Hopkins University, Washington, DC 20001, USA.
Entropy (Basel, Switzerland)
|February 27, 2026
概括
本研究介绍了强化学习的信息理论框架,使用信息瓶原则来改善复杂环境中的探索. 这种新的方法提高了学习效率和解决挑战性路由问题的解决方案质量.
科学领域:
- 人工智能的人工智能
- 机器学习 机器学习
- 计算理论 计算理论
背景情况:
- 当奖励很少时,内在的动机是强化学习 (RL) 探索的关键.
- 对于传统的RL方法来说,在高维状态空间中定义新性是一项挑战.
研究的目的:
- 通过信息瓶原则,为内在动机的RL提出一个信息理论框架.
- 开发一种学习紧的潜态表示的方法,平衡观察压缩和预测信息.
主要方法:
- 利用信息瓶原理创建潜态表示.
- 定义的内在奖励基于潜在空间内的相互信息.
- 使用神经相互信息估计器在高维设置中进行可扩展的估计.
主要成果:
- 拟议的方法证明了勘探效率的提高.
- 与标准RL基线相比,观察到提高了培训稳定性和解决方案质量.
- 有效评估组合路由问题,如TSP和SDVRP.
结论:
- 信息瓶驱动的内在动机为RL探索提供了一个原则性的方法.
- 该框架有效地解决了高维和组合状态空间中的挑战.
- 这种方法推进了内在动机强化学习的最新技术.
相关概念视频
Cognitive Learning
1.5K
Cognitive learning is based on purposive behavior, incidental learning, and insight learning.
E. C. Tolman's theory of purposive behavior emphasizes that much behavior is goal-directed. He argued that to understand behavior, we must look at the entire sequence of actions leading to a goal. For instance, high school students study hard, not just due to past reinforcement but also to achieve the goal of getting into a good college.
Tolman introduced the idea that behavior is influenced by...
E. C. Tolman's theory of purposive behavior emphasizes that much behavior is goal-directed. He argued that to understand behavior, we must look at the entire sequence of actions leading to a goal. For instance, high school students study hard, not just due to past reinforcement but also to achieve the goal of getting into a good college.
Tolman introduced the idea that behavior is influenced by...
1.5K
Incentive Theory: Pull Theory of Motivation
1.0K
Incentive theory, or the "pull theory" of motivation, suggests that external rewards primarily drive behavior. Individuals are motivated to engage in activities when they anticipate a desirable outcome. This is why people often work hard for promotions or study intensively to achieve high grades. These incentives can be tangible, physical rewards such as money or promotions, or intangible, non-physical rewards like praise and social recognition.
The theory differentiates between...
The theory differentiates between...
1.0K
Purposive Learning
545
E. C. Tolman emphasized the purposiveness of behavior — the idea that much of our behavior is goal-directed. For instance, employees who aim for a promotion work diligently to meet their targets. Tolman argued that when classical conditioning and operant conditioning occur, the organism acquires certain expectations. In classical conditioning, a child might fear a dog because they expect it to bite. In operant conditioning, a person might consistently work overtime because they expect a...
545
Reinforcement Schedules
577
Positive reinforcement is a powerful method for teaching new behaviors to both animals and humans. B.F. Skinner demonstrated this with his experiments using rats in a Skinner box. When a rat pressed a lever, it received a food pellet. This immediate reward encouraged the rat to repeat the behavior. This method, where a reward follows every instance of the behavior, is known as continuous reinforcement. It is highly effective for establishing new behaviors quickly.
Once a behavior is learned,...
Once a behavior is learned,...
577
Reinforcement
1.0K
Positive and negative reinforcement are key concepts in operant conditioning, a learning process where the consequences of a behavior affect the likelihood of that behavior being repeated.
Positive reinforcement occurs when a behavior is followed by the presentation of a rewarding stimulus, increasing the frequency of that behavior. For example:
Positive reinforcement occurs when a behavior is followed by the presentation of a rewarding stimulus, increasing the frequency of that behavior. For example:
1.0K
Observational Learning
1.1K
Albert Bandura's observational learning, also known as imitation or modeling, occurs when a person observes and imitates another's behavior. It is a quicker process than operant conditioning. A well-known example is the Bobo doll study, where children who saw an adult acting aggressively towards the doll were more likely to act aggressively when left alone, compared to those who observed a nonaggressive adult. Many psychologists view observational learning as a form of latent learning...
1.1K


