通过自动驾驶的决策转换器进行深度强化学习导航
Lun Ge1, Xiaoguang Zhou1, Yongqiang Li2
1School of Modern Post (School of Automation), Beijing University of Posts and Telecommunications, Beijing, China.
Frontiers in neurorobotics
|April 3, 2024
概括
本研究介绍了通过自动驾驶汽车的决策转换器 (DRLNDT) 算法进行深度强化学习导航. DRLNDT通过使用变压器网络和变异自动编码器,在部分可观测的环境中增强决策.
科学领域:
- 机器人技术 机器人技术 机器人技术
- 人工智能的人工智能
- 计算机视觉 计算机视觉
背景情况:
- 自动驾驶的决策是连续的,基于部分的,杂的环境观察.
- 强化学习 (RL) 通常用于训练从有限数据中获得奖励的代理人.
- 由于部分可观测性和未知的环境模型,城市自主导航面临着挑战.
研究的目的:
- 引入一种新的算法,即通过决策转换器 (DRLNDT) 进行深度强化学习导航,以改善自动驾驶汽车的决策.
- 在部分可观测的城市环境中提高自动驾驶汽车的能力.
- 在现实驾驶场景中解决传感器噪音和堵塞所带来的挑战.
主要方法:
- DRLNDT框架使用软行为者-批判 (SAC) 算法.
- 变压器神经网络模拟了观察和行动中的时间依赖.
- 变化自编码器 (VAE) 从图像中提取潜伏向量,减少状态空间维度,提高训练效率.
主要成果:
- DRLNDT有效地减轻由传感器噪音或阻塞引起的判断错误.
- 多模态状态空间整合了向量状态 (速度,位置) 和图像衍生潜向量.
- 实验表明,在没有先前的环境知识,地图或路线辅助的情况下,DRLNDT可以实现优越的最佳政策.
结论:
- DRLNDT超越了基线和其他缺乏历史数据的政策方法.
- 该算法证明了复杂的城市环境中自动驾驶汽车的增强决策.
- 在部分可观测条件下,DRLNDT提供了强大的导航方法.
相关概念视频
Multi-input and Multi-variable systems
106
Cruise control systems in cars are designed as multi-input systems to maintain a driver's desired speed while compensating for external disturbances such as changes in terrain. The block diagram for a cruise control system typically includes two main inputs: the desired speed set by the driver and any external disturbances, such as the incline of the road. By adjusting the engine throttle, the system maintains the vehicle's speed as close to the desired value as possible.
In the absence...
In the absence...
106
PD Controller: Design
224
In automotive engineering, car suspension systems often employ Proportional Derivative (PD) controllers to enhance performance. PD controllers are utilized to adjust the damping force in response to road conditions. A controller, acting as an amplifier with a constant gain, demonstrates proportional control, with output directly mirroring input.
Designing a continuous-data controller requires selecting and linking components like adders and integrators, which are fundamental in Proportional,...
Designing a continuous-data controller requires selecting and linking components like adders and integrators, which are fundamental in Proportional,...
224
Controller Configurations
96
Controller configurations are crucial in a car's cruise control system because they manage speed over time to maintain a consistent pace regardless of road conditions, thereby meeting design goals. In traditional control systems, fixed-configuration design involves predetermined controller placement. System performance modifications are known as compensation.
Control-system compensation involves various configurations, most commonly series or cascade compensation, in which the controller...
Control-system compensation involves various configurations, most commonly series or cascade compensation, in which the controller...
96
Decision Making
109
Decision-making is a fundamental cognitive process that involves evaluating alternatives and selecting among them. This process can range from simple choices, such as deciding what to wear, to complex decisions, like choosing a major in college or a career path. The complexity of the decision often dictates the approach we use, which can be broadly categorized into two types: automatic and controlled decision-making.
Automatic decision-making is fast, intuitive, and relies on gut feelings...
Automatic decision-making is fast, intuitive, and relies on gut feelings...
109
Control Systems: Applications
606
Electrical engineering plays a pivotal role in our daily lives, with control systems at the heart of many applications, from home appliances to sophisticated space shuttles. Control systems manage and regulate the behavior of devices and processes, ensuring they function safely, correctly, and efficiently.
In modern vehicles, control systems manage various functions to enhance performance and safety. The steering wheel and accelerator are primary inputs in a car's control system. The...
In modern vehicles, control systems manage various functions to enhance performance and safety. The steering wheel and accelerator are primary inputs in a car's control system. The...
606


