A Partial Joint Optimization Algorithm for Autonomous Air Combat Based on Hierarchical Reinforcement Learning.

Area of Science:

• Artificial Intelligence
• Reinforcement Learning
• Autonomous Systems

Background:

• Autonomous air combat strategy design faces challenges like vast exploration spaces and slow decision-making.
• Existing hierarchical methods often train agents independently, limiting dynamic response capabilities.

Purpose of the Study:

• To develop an advanced hierarchical reinforcement learning framework for one-on-one beyond-visual-range (BVR) air combat.
• To enhance exploration efficiency and agent integration across hierarchical levels.
• To improve dynamic response speed and reduce redundant actions in air combat scenarios.

Main Methods:

• Proposed a partial-joint-optimization-based hierarchical (PJOH) learning framework.
• Introduced a time-event dual-driven (TED2) mechanism combining time-driven and event-driven approaches.
• Evaluated the framework in a high-fidelity air combat simulation against state-of-the-art methods.

Main Results:

• The PJOH-TED2 framework demonstrated superior performance compared to four state-of-the-art methods.
• Achieved a win rate of at least 71% in simulated beyond-visual-range (BVR) air combat.
• Secured 1st place in learning methods at the intelligent air game algorithm challenge (IAGAC).

Conclusions:

• The PJOH-TED2 framework effectively addresses exploration and decision-making challenges in autonomous air combat.
• The integration of partial joint optimization and time-event dual-driven mechanisms enhances agent performance and adaptability.
• This approach represents a significant advancement in intelligent game strategies for autonomous air combat.