Related Experiment Video
Updated: Sep 12, 2025

11:53
The Modular Design and Production of an Intelligent Robot Based on a Closed-Loop Control Strategy
Published on: October 14, 2017
11.8K
Fixed-time optimal time-varying formation control for unmanned surface vehicle systems based on reinforcement
Qiaokun Kang1, Qintao Gan1, Ruihong Li1
1Shijiazhuang Campus, Army Engineering University of PLA, Shijiazhuang, 050003, China.
ISA Transactions
|August 9, 2025
Summary
This study introduces a novel reinforcement learning (RL) strategy for unmanned surface vehicle systems (USVSs) to achieve optimal formation control in fixed time, even with unknown system dynamics and unmeasurable states.
Area of Science:
- Robotics and Control Systems
- Artificial Intelligence
- Marine Engineering
Background:
- Unmanned Surface Vehicle Systems (USVSs) require robust control strategies for coordinated operations.
- Partially unmeasurable states and unknown dynamics pose significant challenges in USVS formation control.
- Existing control methods often struggle to guarantee fixed-time convergence and optimal performance simultaneously.
Purpose of the Study:
- To develop a distributed fixed-time optimal time-varying formation control (TVFC) strategy for USVSs.
- To address systems with partially unmeasurable states and unknown dynamics.
- To achieve both formation control and cost optimization objectives.
Main Methods:
- A fixed-time adaptive neural network state observer (FANNSO) was designed to reconstruct unknown dynamics and unmeasurable states.
- A distributed optimization performance index function with exponential terms was proposed.
- A distributed fixed-time optimal TVFC strategy was developed using an actor-critic reinforcement learning structure.
Main Results:
- The proposed control strategy ensures that error signals remain bounded within a fixed time.
- The reinforcement learning algorithm adaptively adjusts the controller for optimal performance.
- Simulation results validated the effectiveness and superiority of the developed method.
Conclusions:
- The novel RL-based TVFC strategy effectively handles USVSs with uncertainties.
- The FANNSO successfully reconstructs system states and dynamics.
- The approach provides a robust solution for fixed-time optimal formation control in USVSs.
Keywords:
Fixed-time controlNeural network observerOptimal formation controlReinforcement learningUnmanned surface vehicleMore Related Videos
Related Concept Videos
Feedback control systems
427
Feedback control systems are categorized in various ways based on their design, analysis, and signal types.
Linear feedback systems are theoretical models that simplify analysis and design. These systems operate under the principle that their output is directly proportional to their input within certain ranges. For instance, an amplifier in a control system behaves linearly as long as the input signal remains within a specific range. However, most physical systems exhibit inherent nonlinearity...
Linear feedback systems are theoretical models that simplify analysis and design. These systems operate under the principle that their output is directly proportional to their input within certain ranges. For instance, an amplifier in a control system behaves linearly as long as the input signal remains within a specific range. However, most physical systems exhibit inherent nonlinearity...
427
Reinforcement Schedules
242
Positive reinforcement is a powerful method for teaching new behaviors to both animals and humans. B.F. Skinner demonstrated this with his experiments using rats in a Skinner box. When a rat pressed a lever, it received a food pellet. This immediate reward encouraged the rat to repeat the behavior. This method, where a reward follows every instance of the behavior, is known as continuous reinforcement. It is highly effective for establishing new behaviors quickly.
Once a behavior is learned,...
Once a behavior is learned,...
242
Time-Domain Interpretation of PD Control
178
Proportional-Derivative (PD) control is a widely used control method in various engineering systems to enhance stability and performance. In a system with only proportional control, common issues include high maximum overshoot and oscillation, observed in both the error signal and its rate of change. This behavior can be divided into three distinct phases: initial overshoot, subsequent undershoot, and gradual stabilization.
Consider the example of control of motor torque. Initially, a positive...
Consider the example of control of motor torque. Initially, a positive...
178
Multi-input and Multi-variable systems
149
Cruise control systems in cars are designed as multi-input systems to maintain a driver's desired speed while compensating for external disturbances such as changes in terrain. The block diagram for a cruise control system typically includes two main inputs: the desired speed set by the driver and any external disturbances, such as the incline of the road. By adjusting the engine throttle, the system maintains the vehicle's speed as close to the desired value as possible.
In the absence...
In the absence...
149
Controller Configurations
149
Controller configurations are crucial in a car's cruise control system because they manage speed over time to maintain a consistent pace regardless of road conditions, thereby meeting design goals. In traditional control systems, fixed-configuration design involves predetermined controller placement. System performance modifications are known as compensation.
Control-system compensation involves various configurations, most commonly series or cascade compensation, in which the controller...
Control-system compensation involves various configurations, most commonly series or cascade compensation, in which the controller...
149
Root-Locus Method
213
A cruise control system in a car is designed to maintain a specified speed automatically by adjusting the gas pedal. The system continuously measures the vehicle's speed and makes fine adjustments to the pedal to achieve this goal. The root locus method is particularly useful for understanding how the cruise control system's behavior changes under varying conditions, such as when the car goes uphill, downhill, or faces strong wind resistance.
This system can be represented by a block...
This system can be represented by a block...
213

