Related Experiment Video
Updated: Nov 23, 2025
07:52
Investigating Motor Skill Learning Processes with a Robotic Manipulandum
Published on: February 12, 2017
9.0K
Modular deep reinforcement learning from reward and punishment for robot navigation.
Jiexin Wang1, Stefan Elfwing2, Eiji Uchibe1
1Department of Brain Robot Interface, ATR Computational Neuroscience Laboratories, 2-2-2 Hikaridai, Seikacho, Soraku-gun, Kyoto 619-0288, Japan.
Summary
This study introduces novel methods for modular reinforcement learning (RL) by dynamically weighting reward and punishment signals. The approach enhances learning efficiency and safety in navigation tasks, outperforming traditional RL techniques.
Area of Science:
- Neuroscience and Artificial Intelligence
- Robotics and Machine Learning
Background:
- Modular Reinforcement Learning (RL) decomposes complex tasks into sub-goals, mirroring animal learning.
- Existing dichotomy-based RL methods (MaxPain, Deep MaxPain) use separate reward and punishment systems but rely on manual weight tuning.
- Manual weighting limits the effective utilization of learned RL modules.
Purpose of the Study:
- To address the limitations of manual weight determination in dichotomy-based RL.
- To propose a state-value dependent weighting scheme for automatic tuning of reward and punishment signals.
- To improve safety and learning efficiency in modular RL tasks, particularly in navigation.
Main Methods:
- Investigated signal scaling of reward and punishment concerning the discounting factor gamma.
- Proposed a weak constraint for signaling design and a state-value dependent weighting scheme (hard-max and softmax) based on Boltzmann distribution analysis.
- Developed a sensor fusion network combining lidar and monocular camera data for enhanced environmental perception.
Main Results:
- The proposed automatic weighting scheme effectively tuned mixing weights, improving module utilization.
- Sensor fusion network demonstrated superior performance compared to single-sensor approaches in simulations and real-world robot experiments.
- The methods showed significant improvements in safety and learning efficiency for maze-solving navigation tasks.
Conclusions:
- Dynamically weighting reward and punishment signals based on state-value is crucial for optimizing modular RL.
- Sensor fusion enhances robustness and performance in robotic navigation tasks.
- The developed approach offers a more efficient and safer alternative to conventional RL methods.
Related Concept Videos
01:23Reinforcement
615
Positive and negative reinforcement are key concepts in operant conditioning, a learning process where the consequences of a behavior affect the likelihood of that behavior being repeated.
Positive reinforcement occurs when a behavior is followed by the presentation of a rewarding stimulus, increasing the frequency of that behavior. For example:
Positive reinforcement occurs when a behavior is followed by the presentation of a rewarding stimulus, increasing the frequency of that behavior. For example:
615
01:24Reinforcement Schedules
330
Positive reinforcement is a powerful method for teaching new behaviors to both animals and humans. B.F. Skinner demonstrated this with his experiments using rats in a Skinner box. When a rat pressed a lever, it received a food pellet. This immediate reward encouraged the rat to repeat the behavior. This method, where a reward follows every instance of the behavior, is known as continuous reinforcement. It is highly effective for establishing new behaviors quickly.
Once a behavior is learned,...
Once a behavior is learned,...
330
01:12Observational Learning
641
Albert Bandura's observational learning, also known as imitation or modeling, occurs when a person observes and imitates another's behavior. It is a quicker process than operant conditioning. A well-known example is the Bobo doll study, where children who saw an adult acting aggressively towards the doll were more likely to act aggressively when left alone, compared to those who observed a nonaggressive adult. Many psychologists view observational learning as a form of latent learning...
641
01:14Avoidance Learning and Learned Helplessness
2.3K
Avoidance learning and learned helplessness are critical concepts in understanding behavioral responses to negative stimuli.
Avoidance learning occurs when an organism learns that a specific behavior can prevent an unpleasant outcome. For example, a student who receives a bad grade may start studying harder to avoid future poor grades. This behavior persists even when the negative outcome is no longer present. Avoidance learning is powerful because it maintains behavior in the absence of the...
Avoidance learning occurs when an organism learns that a specific behavior can prevent an unpleasant outcome. For example, a student who receives a bad grade may start studying harder to avoid future poor grades. This behavior persists even when the negative outcome is no longer present. Avoidance learning is powerful because it maintains behavior in the absence of the...
2.3K
01:22Purposive Learning
302
E. C. Tolman emphasized the purposiveness of behavior — the idea that much of our behavior is goal-directed. For instance, employees who aim for a promotion work diligently to meet their targets. Tolman argued that when classical conditioning and operant conditioning occur, the organism acquires certain expectations. In classical conditioning, a child might fear a dog because they expect it to bite. In operant conditioning, a person might consistently work overtime because they expect a...
302
01:27Associative Learning
882
Associative learning is a fundamental concept in behavioral psychology, wherein a connection is established between two stimuli or events, leading to a learned response. This process is critical in understanding how behaviors are acquired and modified. Conditioning, the mechanism through which associations are formed, can be divided into two main types: classical conditioning and operant conditioning, each elucidating different aspects of associative learning.
Classical conditioning, also known...
Classical conditioning, also known...
882