Research on UCAV Maneuvering Decision Method Based on Heuristic Reinforcement Learning.

Area of Science:

• Robotics and Artificial Intelligence
• Aerospace Engineering
• Military Science

Background:

• Unmanned Combat Aerial Vehicles (UCAVs) are crucial for future air combat superiority.
• Independent UCAV operations require advanced maneuver decision-making capabilities.
• Existing deep reinforcement learning algorithms face challenges in exploration for continuous action spaces.

Purpose of the Study:

• To address the exploration limitations of the Ornstein-Uhlenbeck (OU) strategy in Deep Deterministic Policy Gradient (DDPG) algorithms.
• To propose a novel heuristic DDPG algorithm for UCAV maneuver decision-making.
• To enhance the decision-making performance of UCAVs in air combat scenarios.

Main Methods:

• Development of a UCAV platform model with a continuous action space.
• Introduction of a heuristic exploration strategy to improve the DDPG algorithm.
• Implementation of a UCAV air combat maneuver decision method based on the heuristic DDPG algorithm.

Main Results:

• The proposed heuristic DDPG algorithm demonstrated superior performance compared to existing algorithms in simulated environments.
• The UCAV decision method proved effective across air combat tasks of varying difficulty and attack modes.
• Enhanced exploration capabilities led to more robust and efficient maneuver strategies.

Conclusions:

• The heuristic DDPG algorithm offers a significant advancement for UCAV autonomous decision-making.
• This approach effectively tackles the exploration problem in continuous action spaces for UCAVs.
• The developed method is validated for practical application in UCAV air combat operations.