基于模型的离线增强学习与对抗数据增强
IEEE transactions on neural networks and learning systems
|December 2, 2025
概括
基于模型的离线强化学习 (RL) 使用对抗数据增强来改善政策优化. 通过动态选择模型,MORAL增强了培训数据,从而提高了各种任务的性能.
科学领域:
- 人工智能的人工智能
- 机器学习 机器学习
- 机器人技术 机器人技术 机器人技术
背景情况:
- 基于模型的线下强化学习 (RL) 旨在利用预先收集的数据集优化政策.
- 目前的方法与静态数据和从固定的模型中推断错误作斗争.
- 离线代理无法与数据收集环境进行交互.
研究的目的:
- 引入一种新的方法,即基于模型的离线增强学习与对抗数据增强 (MORAL),以解决离线RL的局限性.
- 通过对抗增强,通过丰富培训数据来增强政策优化.
- 提高基于模型的线下RL的稳定性和适用性.
主要方法:
- MORAL采用对抗性数据增强,用集成模型替代固定视界推出,采用交替采样.
- 一个动态的对抗过程选择组合模型与政策相对应,以减轻乐观偏见.
- 一个差分因子 (DF) 集成用于规范化和误差最小化在外推过程中.
主要成果:
- 实际上,MORAL有效地丰富了培训数据,使得在没有手动推出视野调整的情况下能够进行强有力的政策优化.
- 该方法在各种离线任务中展示了适应性.
- 对D4RL基准的实验表明,MORAL超越了现有的基于模型的线下RL技术.
结论:
- 在基于模型的线下强化学习中,MORAL提供了显著的进步.
- 对抗性数据增强策略提高了政策学习和样本效率.
- MORAL为线下RL挑战提供了一个强大且广泛适用的解决方案.
相关概念视频
Observational Learning
795
Albert Bandura's observational learning, also known as imitation or modeling, occurs when a person observes and imitates another's behavior. It is a quicker process than operant conditioning. A well-known example is the Bobo doll study, where children who saw an adult acting aggressively towards the doll were more likely to act aggressively when left alone, compared to those who observed a nonaggressive adult. Many psychologists view observational learning as a form of latent learning...
795
Associative Learning
1.2K
Associative learning is a fundamental concept in behavioral psychology, wherein a connection is established between two stimuli or events, leading to a learned response. This process is critical in understanding how behaviors are acquired and modified. Conditioning, the mechanism through which associations are formed, can be divided into two main types: classical conditioning and operant conditioning, each elucidating different aspects of associative learning.
Classical conditioning, also known...
Classical conditioning, also known...
1.2K
Reinforcement
791
Positive and negative reinforcement are key concepts in operant conditioning, a learning process where the consequences of a behavior affect the likelihood of that behavior being repeated.
Positive reinforcement occurs when a behavior is followed by the presentation of a rewarding stimulus, increasing the frequency of that behavior. For example:
Positive reinforcement occurs when a behavior is followed by the presentation of a rewarding stimulus, increasing the frequency of that behavior. For example:
791
Avoidance Learning and Learned Helplessness
2.5K
Avoidance learning and learned helplessness are critical concepts in understanding behavioral responses to negative stimuli.
Avoidance learning occurs when an organism learns that a specific behavior can prevent an unpleasant outcome. For example, a student who receives a bad grade may start studying harder to avoid future poor grades. This behavior persists even when the negative outcome is no longer present. Avoidance learning is powerful because it maintains behavior in the absence of the...
Avoidance learning occurs when an organism learns that a specific behavior can prevent an unpleasant outcome. For example, a student who receives a bad grade may start studying harder to avoid future poor grades. This behavior persists even when the negative outcome is no longer present. Avoidance learning is powerful because it maintains behavior in the absence of the...
2.5K
Reinforcement Schedules
436
Positive reinforcement is a powerful method for teaching new behaviors to both animals and humans. B.F. Skinner demonstrated this with his experiments using rats in a Skinner box. When a rat pressed a lever, it received a food pellet. This immediate reward encouraged the rat to repeat the behavior. This method, where a reward follows every instance of the behavior, is known as continuous reinforcement. It is highly effective for establishing new behaviors quickly.
Once a behavior is learned,...
Once a behavior is learned,...
436
Generalization, Discrimination, and Extinction
1.3K
Generalization, discrimination, and extinction are key concepts in operant conditioning that influence how behaviors are learned and maintained.
Generalization occurs when a behavior reinforced in one context is performed in similar situations. For instance, a student who studies diligently for calculus and receives excellent grades might apply the same study habits to psychology and history, expecting similar results. Generalization shows how learning in one setting can influence behavior in...
Generalization occurs when a behavior reinforced in one context is performed in similar situations. For instance, a student who studies diligently for calculus and receives excellent grades might apply the same study habits to psychology and history, expecting similar results. Generalization shows how learning in one setting can influence behavior in...
1.3K
