Autonomous air combat decision making via graph neural networks and reinforcement learning

Area of Science:

• Artificial Intelligence
• Robotics
• Aerospace Engineering

Background:

• Intelligent air combat is a growing field in multi-agent systems due to complex aerial interactions.
• Large-scale air combat presents challenges in scalability, computational complexity, and collaborative decision-making.
• Existing methods struggle with the dynamic and uncertain nature of these environments.

Purpose of the Study:

• To propose a novel autonomous decision-making approach for multi-aircraft systems.
• To address scalability and decision-making challenges in large-scale air combat.
• To enhance collaborative decision-making in dynamic and uncertain environments.

Main Methods:

• Developed a multi-aircraft autonomous decision-making approach using graphs and multi-agent reinforcement learning (MADRL) under zero-order optimization.
• Implemented the GraphZero-PPO algorithm, integrating GraphSAGE and zero-order optimization.
• Utilized graph structures for adaptability to high dynamics and high-dimensional characteristics of multi-agent systems.

Main Results:

• The GraphZero-PPO algorithm demonstrated effective adaptation to large-scale air combat environments.
• Achieved high win rates in both 1v1 and 8v8 simulation scenarios.
• Showcased rapid decision-making performance, particularly for missile launches.

Conclusions:

• The proposed graph-based MADRL approach effectively tackles challenges in large-scale intelligent air combat.
• GraphZero-PPO offers a promising solution for enhancing autonomous decision-making and combat effectiveness.
• The method's adaptability and efficiency are key contributions to the field.