通过使用基于图形神经网络的内在奖励机制,加强学习的增强探索
J Arun Pandian1, Ramkumar Thirunavukarasu2, Rajganesh Nagarajan3
1School of Computer Science Engineering and Information Systems, Vellore Institute of Technology, Vellore, India. arunpandian.j@vit.ac.in.
Scientific reports
|November 14, 2025
概括
本研究介绍了基于图形神经网络的内在奖励学习 (GNN-IRL),以改善强化学习 (RL) 的探索. 通过识别新状态,GNN-IRL增强了代理学习,在效率和覆盖率方面表现优于现有的方法.
科学领域:
- 人工智能的人工智能
- 机器学习 机器学习
- 强化学习是一种强化学习.
背景情况:
- 探索-开发困境是强化学习 (RL) 中的一个关键挑战,影响了代理学习效率.
- 现有的内在奖励策略往往难以有效地引导代理人进入新的状态.
研究的目的:
- 提出一个新的框架,基于图形神经网络的内在奖励学习 (GNN-IRL),以解决RL的勘探-开发权衡问题.
- 利用图形神经网络 (GNN) 来建模状态转换和关系以计算内在奖励.
主要方法:
- 开发了一个GNN-IRL框架,利用GNN来表示环境状态及其关系.
- 基于图表中心性和逆度分析计算的内在奖励,以优先考虑新状态探索.
- 离散的连续状态变量来构建GNN-IRL实现的状态图.
主要成果:
- 在四个基准环境 (CartPole-v1,MountainCar-v0,Taxi-v3,LunarLander-v3) 中,GNN-IRL表现出了卓越的性能.
- 与最先进的方法相比,该框架显著提高了融合率,累积奖励,勘探效率和国家覆盖率.
- GNN-IRL有效地平衡了勘探和开发,提高了样本效率并加速了政策学习.
结论:
- GNN-IRL提供了一种有效的方法来增强在离散强化学习领域的探索.
- 基于GNN的内在奖励机制成功地引导代理商更有效地探索新状态.
- 这一框架显示了提高RL应用中的样本效率和学习速度的前景.
相关概念视频
01:23Reinforcement
804
Positive and negative reinforcement are key concepts in operant conditioning, a learning process where the consequences of a behavior affect the likelihood of that behavior being repeated.
Positive reinforcement occurs when a behavior is followed by the presentation of a rewarding stimulus, increasing the frequency of that behavior. For example:
Positive reinforcement occurs when a behavior is followed by the presentation of a rewarding stimulus, increasing the frequency of that behavior. For example:
804
01:24Reinforcement Schedules
440
Positive reinforcement is a powerful method for teaching new behaviors to both animals and humans. B.F. Skinner demonstrated this with his experiments using rats in a Skinner box. When a rat pressed a lever, it received a food pellet. This immediate reward encouraged the rat to repeat the behavior. This method, where a reward follows every instance of the behavior, is known as continuous reinforcement. It is highly effective for establishing new behaviors quickly.
Once a behavior is learned,...
Once a behavior is learned,...
440
01:12Observational Learning
804
Albert Bandura's observational learning, also known as imitation or modeling, occurs when a person observes and imitates another's behavior. It is a quicker process than operant conditioning. A well-known example is the Bobo doll study, where children who saw an adult acting aggressively towards the doll were more likely to act aggressively when left alone, compared to those who observed a nonaggressive adult. Many psychologists view observational learning as a form of latent learning...
804
01:23Primary and Secondary Reinforcers
818
In psychology, reinforcement is a key concept in behavior modification. B.F. Skinner demonstrated this with his experiments involving rats in what is known as a Skinner box. The rats learned to press a lever to receive food, a primary reinforcer that fulfilled their innate need for nourishment.
Effective reinforcers for humans vary depending on the individual and the context. Primary reinforcers, such as food, water, sleep, shelter, and pleasure, have inherent value and satisfy basic biological...
Effective reinforcers for humans vary depending on the individual and the context. Primary reinforcers, such as food, water, sleep, shelter, and pleasure, have inherent value and satisfy basic biological...
818
01:18Incentive Theory: Pull Theory of Motivation
840
Incentive theory, or the "pull theory" of motivation, suggests that external rewards primarily drive behavior. Individuals are motivated to engage in activities when they anticipate a desirable outcome. This is why people often work hard for promotions or study intensively to achieve high grades. These incentives can be tangible, physical rewards such as money or promotions, or intangible, non-physical rewards like praise and social recognition.
The theory differentiates between...
The theory differentiates between...
840
01:25Neural Circuits
2.6K
Neural circuits and neuronal pools are two of the main structures found in the nervous system. Neural circuits are networks of neurons that work together to carry out a specific task or process. They consist of interconnected neurons and glial cells, which provide structural and metabolic support.
Neuronal pools are collections of nerve cells with similar functions and interact through chemical and electrical signals. These pools include both interneurons (the central neural circuit nodes that...
Neuronal pools are collections of nerve cells with similar functions and interact through chemical and electrical signals. These pools include both interneurons (the central neural circuit nodes that...
2.6K