Related Experiment Video
Updated: Jul 25, 2025
08:59
An Open-Source Virtual Reality System for the Measurement of Spatial Learning in Head-Restrained Mice
Published on: March 3, 2023
2.1K
Spatial-temporal recurrent reinforcement learning for autonomous ships
1Technische Universität Dresden, Chair of Econometrics and Statistics, esp. in the Transport Sector, Dresden, 01062, Germany.
Summary
This study introduces a novel deep Q-network for autonomous ship steering, enhancing collision avoidance and multi-ship navigation. The approach demonstrates robust performance in complex maritime scenarios.
Area of Science:
- Artificial Intelligence
- Robotics
- Maritime Engineering
Background:
- Autonomous ship navigation requires advanced decision-making capabilities to handle complex, dynamic maritime environments.
- Existing methods often struggle with multi-target scenarios and partial observability, limiting their real-world applicability.
- Ensuring safety through collision avoidance and adherence to maritime regulations is paramount.
Purpose of the Study:
- To propose a spatial-temporal recurrent neural network architecture for deep Q-networks (DQN) to enhance autonomous ship steering.
- To develop a robust system capable of managing multiple surrounding ships and partial observability.
- To integrate a state-of-the-art collision risk metric and International Regulations for Preventing Collisions at Sea (COLREGs) into the agent's decision-making process.
Main Methods:
- Development of a spatial-temporal recurrent neural network integrated with deep Q-networks.
- Inclusion of a novel collision risk metric for situation assessment.
- Incorporation of COLREG rules into the reward function design.
- Validation using 'Around the Clock' and Imazu (1987) multi-ship encounter datasets.
Main Results:
- The proposed DQN architecture effectively steers autonomous ships in complex scenarios.
- The system demonstrates robustness in handling an arbitrary number of target ships and partial observability.
- Performance comparisons show superiority over artificial potential field and velocity obstacle methods in maritime path planning.
- The architecture proves compatible with other deep reinforcement learning algorithms, including actor-critic frameworks.
Conclusions:
- The novel spatial-temporal DQN architecture offers a significant advancement in autonomous ship path planning and collision avoidance.
- The approach provides a robust and adaptable solution for multi-agent maritime scenarios.
- This work paves the way for safer and more efficient autonomous maritime operations.
Related Concept Videos
01:12Observational Learning
222
Albert Bandura's observational learning, also known as imitation or modeling, occurs when a person observes and imitates another's behavior. It is a quicker process than operant conditioning. A well-known example is the Bobo doll study, where children who saw an adult acting aggressively towards the doll were more likely to act aggressively when left alone, compared to those who observed a nonaggressive adult. Many psychologists view observational learning as a form of latent learning...
222
01:24Reinforcement Schedules
212
Positive reinforcement is a powerful method for teaching new behaviors to both animals and humans. B.F. Skinner demonstrated this with his experiments using rats in a Skinner box. When a rat pressed a lever, it received a food pellet. This immediate reward encouraged the rat to repeat the behavior. This method, where a reward follows every instance of the behavior, is known as continuous reinforcement. It is highly effective for establishing new behaviors quickly.
Once a behavior is learned,...
Once a behavior is learned,...
212
01:23Reinforcement
289
Positive and negative reinforcement are key concepts in operant conditioning, a learning process where the consequences of a behavior affect the likelihood of that behavior being repeated.
Positive reinforcement occurs when a behavior is followed by the presentation of a rewarding stimulus, increasing the frequency of that behavior. For example:
Positive reinforcement occurs when a behavior is followed by the presentation of a rewarding stimulus, increasing the frequency of that behavior. For example:
289
01:27Associative Learning
461
Associative learning is a fundamental concept in behavioral psychology, wherein a connection is established between two stimuli or events, leading to a learned response. This process is critical in understanding how behaviors are acquired and modified. Conditioning, the mechanism through which associations are formed, can be divided into two main types: classical conditioning and operant conditioning, each elucidating different aspects of associative learning.
Classical conditioning, also known...
Classical conditioning, also known...
461
01:05Instinctive Drift
254
Instinctive drift refers to the tendency of animals to revert to their innate behaviors despite repeated reinforcement. Breland and Breland demonstrated this concept in an experiment with a raccoon. The raccoon was trained to pick up two coins and place them in a container in exchange for food. Initially, the raccoon learned to associate the coins with food, making them a conditioned stimulus or a substitute for food. However, over time, the raccoon became less willing to put the coins into the...
254
01:14Avoidance Learning and Learned Helplessness
1.8K
Avoidance learning and learned helplessness are critical concepts in understanding behavioral responses to negative stimuli.
Avoidance learning occurs when an organism learns that a specific behavior can prevent an unpleasant outcome. For example, a student who receives a bad grade may start studying harder to avoid future poor grades. This behavior persists even when the negative outcome is no longer present. Avoidance learning is powerful because it maintains behavior in the absence of the...
Avoidance learning occurs when an organism learns that a specific behavior can prevent an unpleasant outcome. For example, a student who receives a bad grade may start studying harder to avoid future poor grades. This behavior persists even when the negative outcome is no longer present. Avoidance learning is powerful because it maintains behavior in the absence of the...
1.8K