A hierarchical reinforcement learning method for missile evasion and guidance

Area of Science:

• Aerospace Engineering
• Artificial Intelligence
• Control Systems

Background:

• Missile guidance and evasion are critical for modern warfare.
• Traditional methods often struggle with complex, dynamic scenarios.
• Reinforcement learning offers a potential solution for adaptive control.

Purpose of the Study:

• To develop a hierarchical reinforcement learning algorithm for simultaneous missile guidance and evasion.
• To improve missile performance in terms of accuracy, time, and energy consumption.
• To create a robust agent adaptable to autopilot lag and measurement noise.

Main Methods:

• Proposed a hierarchical proximal policy optimization (PPO) reinforcement learning algorithm.
• Implemented a two-layer agent structure: a high-level policy selector and low-level guidance/evasion agents.
• Defined distinct reward functions considering guidance accuracy, flight time, energy consumption, and field-of-view constraints.

Main Results:

• The hierarchical PPO algorithm achieved a 100% success rate on a test dataset.
• The agent demonstrated adaptability and robustness against autopilot lag and measurement noise.
• Reinforcement learning guidance law showed superior guidance accuracy, reduced average time, and lower energy consumption compared to traditional laws.

Conclusions:

• Hierarchical reinforcement learning is effective for complex missile maneuvering tasks.
• The proposed algorithm significantly enhances missile performance over traditional guidance laws.
• The agent's robustness and adaptability make it suitable for real-world applications.