A Multi-Dimensional Goal Aircraft Guidance Approach Based on Reinforcement Learning with a Reward Shaping Algorithm

Area of Science:

• Aerospace Engineering
• Artificial Intelligence
• Control Systems

Background:

• Aircraft guidance to 4D waypoints presents a complex multi-dimensional control challenge.
• Current air traffic control (ATC) simulators face limitations in managing direct aircraft approach patterns.

Purpose of the Study:

• To propose and evaluate a multi-layer reinforcement learning (RL) approach for solving the multi-dimensional goal aircraft guidance problem.
• To enhance the performance and efficiency of aircraft guidance systems within ATC simulators.

Main Methods:

• A multi-layer RL method is developed to simplify neural network architecture and reduce state dimensions.
• A shaped reward function incorporating potential functions and the Dubins path method is applied.
• The approach assists autopilot systems in guiding aircraft to specified 4D waypoints (latitude, longitude, altitude, heading, arrival time).

Main Results:

• Simulation results demonstrate significant improvements in convergence efficiency compared to existing methods.
• The proposed approach shows enhanced trajectory performance for aircraft guidance.
• Experimental validation confirms the effectiveness of the multi-layer RL strategy.

Conclusions:

• The multi-layer RL approach effectively addresses the complexities of 4D waypoint guidance.
• The method offers potential applications in team aircraft guidance, enabling direct goal approaches.
• This research overcomes limitations in current ATC simulators by facilitating more efficient aircraft routing.