Accelerate Deep Reinforcement Learning for Aerial Combat With Automated Engagement Curriculum Learning

Deep reinforcement learning (DRL) has emerged as a transformative paradigm for autonomous uncrewed combat aerial vehicle (UCAV) development. However, the application of DRL to complex aerial combat scenarios remains fundamentally challenging due to the combinatorial explosion of state-action spaces and the inherently adversarial dynamics of air engagements. These characteristics lead to prohibitive sample complexity and unstable convergence during training. To overcome these limitations, we present the automated engagement curriculum learning (AECL) framework, which introduces two principal innovations: 1) a combat complexity measurer that quantitatively assesses engagement scenarios and 2) an engagement-aware training scheduler that dynamically modulates task difficulty. Departing from conventional curriculum learning (CL) approaches constrained by fixed scenario sequences, AECL's closed-loop adaptation mechanism enables continuous optimization of the training progression. Comprehensive experimental evaluations demonstrate AECL's superior performance across multiple metrics, reducing the required training samples by 64.8% and online training time by 61.9%, while consistently outperforming standard DRL baselines in win rate performance. Furthermore, AECL-trained agents exhibit significantly enhanced tactical decision-making capabilities and more robust policy execution in operational environments. These results substantiate AECL's dual capability to substantially reduce computational requirements while producing highly competent UCAV agents capable of effective operation in dynamic, high-threat combat scenarios.