Related Experiment Video
Updated: Dec 4, 2025
06:57
Pavlovian Conditioned Approach Training in Rats
Published on: February 4, 2016
11.3K
Non-action Learning: Saving Action-Associated Cost Serves as a Covert Reward.
Sai Tanimoto1, Masashi Kondo1, Kenji Morita2,3
1Department of Physiology, Graduate School of Medicine, The University of Tokyo, Tokyo, Japan.
Frontiers in Behavioral Neuroscience
|October 26, 2020
Summary
Mice learn to "do nothing" by associating inaction with covert rewards, like saving energy. This challenges traditional reinforcement learning models that focus only on overt rewards.
Area of Science:
- Neuroscience
- Decision-making
- Animal behavior
Background:
- Reinforcement learning models explain choices based on overt rewards and punishments.
- Traditional models struggle to account for decisions made without explicit external feedback.
Purpose of the Study:
- To investigate if "doing nothing" can be an actively chosen behavior reinforced by "covert rewards" such as energy conservation.
- To explore decision-making in mice using a task involving varying probabilities of reward.
Main Methods:
- Mice were trained to pull a lever for water rewards with different probabilities associated with auditory cues.
- Behavioral data on lever-pulling choices were analyzed.
- Standard Q-learning models were compared against modified models incorporating covert rewards for inaction.
Main Results:
- Mice decreased lever-pulling for lower reward probabilities and increased it for higher probabilities.
- Standard Q-learning models failed to predict this behavior.
- A modified Q-learning model, which included a "covert reward" for not pulling the lever, accurately explained the observed decision-making patterns.
Conclusions:
- "Doing nothing" can be a deliberate choice reinforced by internal, unstated benefits.
- Covert rewards, such as saving energy, play a significant role in decision-making processes.
- This finding suggests a need to expand reinforcement learning frameworks to include implicit reward mechanisms.
Related Concept Videos
01:22Purposive Learning
321
E. C. Tolman emphasized the purposiveness of behavior — the idea that much of our behavior is goal-directed. For instance, employees who aim for a promotion work diligently to meet their targets. Tolman argued that when classical conditioning and operant conditioning occur, the organism acquires certain expectations. In classical conditioning, a child might fear a dog because they expect it to bite. In operant conditioning, a person might consistently work overtime because they expect a...
321
01:21Cognitive Learning
889
Cognitive learning is based on purposive behavior, incidental learning, and insight learning.
E. C. Tolman's theory of purposive behavior emphasizes that much behavior is goal-directed. He argued that to understand behavior, we must look at the entire sequence of actions leading to a goal. For instance, high school students study hard, not just due to past reinforcement but also to achieve the goal of getting into a good college.
Tolman introduced the idea that behavior is influenced by...
E. C. Tolman's theory of purposive behavior emphasizes that much behavior is goal-directed. He argued that to understand behavior, we must look at the entire sequence of actions leading to a goal. For instance, high school students study hard, not just due to past reinforcement but also to achieve the goal of getting into a good college.
Tolman introduced the idea that behavior is influenced by...
889
01:27Associative Learning
924
Associative learning is a fundamental concept in behavioral psychology, wherein a connection is established between two stimuli or events, leading to a learned response. This process is critical in understanding how behaviors are acquired and modified. Conditioning, the mechanism through which associations are formed, can be divided into two main types: classical conditioning and operant conditioning, each elucidating different aspects of associative learning.
Classical conditioning, also known...
Classical conditioning, also known...
924
01:12Observational Learning
688
Albert Bandura's observational learning, also known as imitation or modeling, occurs when a person observes and imitates another's behavior. It is a quicker process than operant conditioning. A well-known example is the Bobo doll study, where children who saw an adult acting aggressively towards the doll were more likely to act aggressively when left alone, compared to those who observed a nonaggressive adult. Many psychologists view observational learning as a form of latent learning...
688
01:14Avoidance Learning and Learned Helplessness
2.3K
Avoidance learning and learned helplessness are critical concepts in understanding behavioral responses to negative stimuli.
Avoidance learning occurs when an organism learns that a specific behavior can prevent an unpleasant outcome. For example, a student who receives a bad grade may start studying harder to avoid future poor grades. This behavior persists even when the negative outcome is no longer present. Avoidance learning is powerful because it maintains behavior in the absence of the...
Avoidance learning occurs when an organism learns that a specific behavior can prevent an unpleasant outcome. For example, a student who receives a bad grade may start studying harder to avoid future poor grades. This behavior persists even when the negative outcome is no longer present. Avoidance learning is powerful because it maintains behavior in the absence of the...
2.3K
01:28Behaviorism
4.1K
The field of behaviorism was pioneered by figures such as Ivan Pavlov, John B. Watson, and B.F. Skinner fundamentally shifted the focus of psychology to the observable and controllable aspects of human and animal behavior. This shift marked a critical evolution in the discipline, emphasizing scientific rigor and experimental methodology.
The core premise of behaviorism is its focus on observable behavior rather than internal thoughts or feelings. This approach argues that true scientific...
The core premise of behaviorism is its focus on observable behavior rather than internal thoughts or feelings. This approach argues that true scientific...
4.1K