奖励设计和超参数调整用于自主赛车中的可泛化深度强化学习代理
Naga Sai Shreya Kunda1, Pranave Kc1, Mayank Pandey1
1Department of Computer Science and Engineering, Amrita School of Computing, Amrita Vishwa Vidyapeetham, Bengaluru, India.
Scientific reports
|December 16, 2025
概括
在深度强化学习 (DRL) 中优化奖励系统和超参数可以显著提高自主赛车的性能. 精心调整的近距离政策优化 (PPO) 代理人在多种多样的未见的轨道上有效地泛化.
科学领域:
- 自主系统 自主系统
- 机器学习 机器学习
- 机器人技术 机器人技术 机器人技术
背景情况:
- 深度强化学习 (DRL) 使自主代理能够实时做出决策.
- 跨多种赛道的通用化仍然是自主赛车的重大挑战.
- AWS DeepRacer 平台作为DRL在赛车中的基准.
研究的目的:
- 调查奖励系统设计和超参数调整对自动赛车DRL代理的影响.
- 在不同的奖励结构下比较近距离政策优化 (PPO) 和软行为者批判 (SAC) 算法.
- 为了确定最佳的超参数,以提高性能和通用化.
主要方法:
- 在AWS DeepRacer平台上对PPO和SAC算法进行了比较分析.
- 调整了关键的超参数,包括批量大小,学习率,折扣因子和.
- 评估了两种不同的奖励结构和21个未见的轨道的代理业绩.
主要成果:
- 一个精心设计的奖励机制与优化的超参数 (批量大小128,学习率0.0003,折扣系数0.99,率0.01) 能够使PPO实现平均环路时间12.464秒.
- 与标准基准相比,优化的PPO代理在未见的轨道上展示了卓越的性能和概括能力.
- 超参数灵敏度分析强调了调整在大型DRL系统中的关键作用.
结论:
- 有效的奖励塑造和超参数优化对于在基于DRL的自动赛车中实现高性能和通用性至关重要.
- 该研究为开发强大的自主赛车代理提供了经过验证的方法.
- 这些发现强调了DRL在现实世界自主场景中的实际应用性.
相关概念视频
01:23Reinforcement
786
Positive and negative reinforcement are key concepts in operant conditioning, a learning process where the consequences of a behavior affect the likelihood of that behavior being repeated.
Positive reinforcement occurs when a behavior is followed by the presentation of a rewarding stimulus, increasing the frequency of that behavior. For example:
Positive reinforcement occurs when a behavior is followed by the presentation of a rewarding stimulus, increasing the frequency of that behavior. For example:
786
01:17Rolling Resistance: Problem Solving
766
Rolling resistance, also known as rolling friction, is the force that resists the motion of a rolling object, such as a wheel, tire, or ball, when it moves over a surface. It is caused by the deformation of the object and the surface in contact with each other, as well as other factors like internal friction, hysteresis, and energy losses within the materials. Rolling resistance opposes the object's motion, requiring additional energy to overcome it and maintain movement. In practical...
766
01:24Reinforcement Schedules
436
Positive reinforcement is a powerful method for teaching new behaviors to both animals and humans. B.F. Skinner demonstrated this with his experiments using rats in a Skinner box. When a rat pressed a lever, it received a food pellet. This immediate reward encouraged the rat to repeat the behavior. This method, where a reward follows every instance of the behavior, is known as continuous reinforcement. It is highly effective for establishing new behaviors quickly.
Once a behavior is learned,...
Once a behavior is learned,...
436