An Intelligent Maneuver Decision-Making Approach for Air Combat Based on Deep Reinforcement Learning and Transformer...

Area of Science:

• Artificial Intelligence
• Robotics
• Control Systems

Background:

• Traditional maneuver decision-making falters with incomplete situational data in complex electromagnetic environments.
• Reliance on accurate information limits traditional methods when opponent data is missing.

Purpose of the Study:

• To develop an autonomous maneuver decision-making approach robust to missing opponent information.
• To enhance decision-making accuracy in dynamic and uncertain environments using deep reinforcement learning.

Main Methods:

• Integration of a Transformer network within a deep reinforcement learning (DRL) architecture for actor and critic networks.
• Leveraging Transformer's ability to identify dependencies in time-series trajectory data to compensate for information loss.
• Implementation of an effective decision-making reward, prioritized sampling, and dynamic learning rate adjustment to address DRL training challenges.

Main Results:

• The proposed DRL approach with Transformer integration significantly outperforms traditional DRL algorithms.
• Demonstrated higher win rates in simulations involving missing opponent information.
• Improved stability and efficiency in agent training despite the introduction of the Transformer network.

Conclusions:

• The developed autonomous maneuver decision-making approach effectively handles missing opponent information.
• Transformer-enhanced DRL offers a promising solution for robust decision-making in complex, uncertain environments.
• The proposed training enhancements ensure stable and efficient learning for the DRL agent.