无人机飞行控制的强化学习:评估固定翼无人机的持续空间强化学习算法
Hasan Raza Khanzada1, Adnan Maqsood1, Abdul Basit2
1School of Interdisciplinary Engineering and Sciences (SINES), National University of Science and Technology, Islamabad, Pakistan.
PloS one
|October 9, 2025
概括
强化学习 (RL) 提高了固定翼无人机 (UAV) 的飞行控制. 与其他RL方法和PID控制器相比,软行动者-关键 (SAC) 算法表现出卓越的稳定性和响应性.
科学领域:
- 机器人和控制系统 机器人和控制系统
- 航空航天领域的人工智能
- 自主系统工程 自主系统工程
背景情况:
- 飞行控制系统正在越来越多地整合强化学习 (RL) 以提高性能.
- 现有研究强调RL在控制固定翼无人机 (UAV) 的潜力.
- 在用于无人机飞行控制的连续空间RL算法的比较分析中存在一个差距.
研究的目的:
- 对固定翼无人机飞行控制的领先连续空间RL算法进行比较分析.
- 评估深度决定性政策梯度 (DDPG),双延迟深度决定性政策梯度 (TD3),近距离政策优化 (PPO),信任区域政策优化 (TRPO) 和软行为者-批评 (SAC) 算法的适用性.
- 在动态和不确定的环境中评估算法性能,考虑无人机俯冲,滚动和方向控制.
主要方法:
- 在高保真模拟环境中训练了五个突出的RL算法 (DDPG,TD3,PPO,TRPO,SAC).
- 在不同的条件和环境干扰下 (例如,风暴) 评估无人机飞行控制性能.
- 与经典的比例整数导数 (PID) 控制器进行RL算法性能比较.
主要成果:
- 在稳定性,响应性和强度方面,RL算法显著超过PID控制器.
- 软演员-批评 (SAC) 算法在400集内实现了趋同.
- SAC保持稳定状态误差低于3%,这表明在评估的RL算法中,性能与收之间的最佳权衡.
结论:
- 与传统方法相比,强化学习为固定翼无人机提供了更好的飞行控制方法.
- 软演员-关键 (SAC) 算法为复杂的无人机飞行动态提供了高度有效和高效的解决方案.
- 这项研究为选择和集成最佳RL算法到先进的无人机控制系统提供了关键的见解.
相关概念视频
01:26Feedback control systems
685
Feedback control systems are categorized in various ways based on their design, analysis, and signal types.
Linear feedback systems are theoretical models that simplify analysis and design. These systems operate under the principle that their output is directly proportional to their input within certain ranges. For instance, an amplifier in a control system behaves linearly as long as the input signal remains within a specific range. However, most physical systems exhibit inherent nonlinearity...
Linear feedback systems are theoretical models that simplify analysis and design. These systems operate under the principle that their output is directly proportional to their input within certain ranges. For instance, an amplifier in a control system behaves linearly as long as the input signal remains within a specific range. However, most physical systems exhibit inherent nonlinearity...
685
01:29Application of Linearization and Approximation
37
A drone flying through complex terrain often relies on more than one sensing method to estimate small changes in altitude. Along with direct measurements, air pressure provides a useful indirect indicator of vertical movement. Atmospheric pressure decreases as altitude increases, and this relationship is commonly described using an exponential model. Although accurate, converting pressure measurements into altitude values requires calculations that are too complex to perform repeatedly during...
37
01:17Open and closed-loop control systems
1.6K
Control systems are foundational elements in automation and engineering. They are broadly categorized into open-loop and closed-loop systems. These classifications hinge on the presence or absence of feedback mechanisms, significantly influencing the system's performance, complexity, and application.
An open-loop control system operates without feedback from the output. It consists of two primary elements: the controller and the controlled process. The controller receives an input signal...
An open-loop control system operates without feedback from the output. It consists of two primary elements: the controller and the controlled process. The controller receives an input signal...
1.6K
01:24Absolute Motion Analysis- General Plane Motion
536
Visualize a drone, with its propellers spinning rapidly, hovering mid-air. The fascinating movements and operations of this drone can be comprehended by applying the principle of general plane motion.
As the drone's propellers rotate, an upward force is generated that counteracts the force of gravity, enabling the drone to lift off from the ground. This initial movement of the drone is along a straight path, representing a form of translational motion. In this phase, every point on the...
As the drone's propellers rotate, an upward force is generated that counteracts the force of gravity, enabling the drone to lift off from the ground. This initial movement of the drone is along a straight path, representing a form of translational motion. In this phase, every point on the...
536
01:22Controller Configurations
352
Controller configurations are crucial in a car's cruise control system because they manage speed over time to maintain a consistent pace regardless of road conditions, thereby meeting design goals. In traditional control systems, fixed-configuration design involves predetermined controller placement. System performance modifications are known as compensation.
Control-system compensation involves various configurations, most commonly series or cascade compensation, in which the controller...
Control-system compensation involves various configurations, most commonly series or cascade compensation, in which the controller...
352
01:24Reinforcement Schedules
458
Positive reinforcement is a powerful method for teaching new behaviors to both animals and humans. B.F. Skinner demonstrated this with his experiments using rats in a Skinner box. When a rat pressed a lever, it received a food pellet. This immediate reward encouraged the rat to repeat the behavior. This method, where a reward follows every instance of the behavior, is known as continuous reinforcement. It is highly effective for establishing new behaviors quickly.
Once a behavior is learned,...
Once a behavior is learned,...
458