Reinforcement Learning-Based Formation Control for Uncrewed Surface Vehicles Under Aperiodic DoS Attacks: A...

Area of Science:

• Robotics
• Control Systems
• Cybersecurity

Background:

• Distributed formation control for uncrewed surface vehicles (USVs) faces challenges from cyber-attacks like denial-of-service (DoS).
• Ensuring system stability and reliable data communication is crucial for coordinated USV operations.

Purpose of the Study:

• To design a robust distributed formation control framework for USVs against aperiodic DoS attacks.
• To achieve convergence to a Stackelberg-Nash equilibrium (SNE) using reinforcement learning.
• To enhance system resilience through data reconstruction methods.

Main Methods:

• A Stackelberg-Nash game (SNG) framework is employed to model the strategic interactions.
• An actor-critic (AC) reinforcement learning (RL) algorithm is utilized for online policy approximation.
• A consensus-based estimator is developed to reconstruct missing neighbor data.

Main Results:

• The AC-RL algorithm converges to the SNE, enabling effective formation control.
• The consensus-based estimator ensures data integrity despite communication interruptions.
• Lyapunov-based analysis confirms the input-to-state stability (ISS) of the estimator and SGUUB stability of the closed-loop system.

Conclusions:

• The proposed framework effectively achieves accurate trajectory tracking for USVs.
• The system demonstrates significant robustness against frequent DoS attacks.
• The integration of RL and consensus estimation provides a viable solution for secure USV formation control.