An LEO Constellation Early Warning System Decision-Making Method Based on Hierarchical Reinforcement Learning.

Area of Science:

• Aerospace Engineering
• Artificial Intelligence
• Space Systems

Background:

• Tracking highly maneuverable hypersonic vehicles presents significant challenges for Low Earth Orbit (LEO) constellations in space-based early warning systems.
• Uncertain trajectories of hypersonic vehicles demand advanced autonomous decision-making capabilities for multi-satellite cooperative positioning.

Purpose of the Study:

• To develop an intelligent autonomous decision-making algorithm for LEO constellations to achieve cooperative geometric positioning of hypersonic glide vehicles (HGVs).
• To enhance the cooperative positioning accuracy and resource management of LEO satellite constellations for early warning applications.

Main Methods:

• Designed a novel algorithm, Multi-Agent Proximal Policy Optimization with Random Hill Climbing (MAPPO-RHC), integrating hierarchical reinforcement learning and local search.
• Employed hierarchical decision-making to reduce solution space and maximize global reward, coupled with random hill climbing for enhanced solution space exploration.
• Simulated and analyzed the performance of MAPPO-RHC in two scenarios, comparing it against four existing intelligent satellite decision-making algorithms.

Main Results:

• The MAPPO-RHC algorithm demonstrated superior performance in achieving balanced resource allocation among satellites.
• MAPPO-RHC significantly improved geometric positioning accuracy compared to other intelligent decision-making algorithms.
• Simulation results validated the algorithm's effectiveness in complex tracking scenarios.

Conclusions:

• The MAPPO-RHC algorithm offers a feasible and effective solution for real-time decision-making in LEO constellation early warning systems.
• The proposed approach enhances the capability of LEO constellations to autonomously track and position hypersonic threats with high accuracy.