Related Experiment Video
Updated: Oct 23, 2025
11:15
Automated Visual Cognitive Tasks for Recording Neural Activity Using a Floor Projection Maze
Published on: February 20, 2014
13.3K
Linear reinforcement learning in planning, grid fields, and cognitive control
Payam Piray1, Nathaniel D Daw2
1Princeton Neuroscience Institute, Princeton University, Princeton, NJ, USA. ppiray@princeton.edu.
Nature Communications
|August 17, 2021
Summary
This study presents a new brain model for decision-making, enabling flexible planning by reusing future event maps. This model offers a quantifiable understanding of cognitive control and biases in flexible replanning.
Area of Science:
- Neuroscience
- Computational Neuroscience
- Cognitive Science
Background:
- Flexible planning in the brain relies on reusing past computations.
- Inappropriate reuse of computations can lead to inflexibilities like habits and compulsions.
- A comprehensive model for these processes is currently lacking.
Purpose of the Study:
- To introduce a novel model for decision-making in the brain.
- To explain how flexible choice arises from reusing temporally abstracted maps of future events.
- To provide a quantifiable account of biases associated with flexible planning.
Main Methods:
- Developed a model inspired by control engineering principles.
- Replaced traditional nonlinear optimization with a linear approximation.
- Modeled decision-making as a soft maximization around a default policy.
Main Results:
- The model enables biologically-realistic, flexible choice.
- It accounts for specific, quantifiable biases in decision-making.
- Demonstrates links between flexible replanning, cognitive control, and behavioral neuroscience phenomena.
Conclusions:
- The model offers insights into stable, componential representation of long-distance contingencies in the brain.
- Explains how the brain can utilize these representations for choice guidance under changing goals.
- Connects flexible planning, cognitive control, and the neural basis of decision-making.
Related Concept Videos
01:21Cognitive Learning
720
Cognitive learning is based on purposive behavior, incidental learning, and insight learning.
E. C. Tolman's theory of purposive behavior emphasizes that much behavior is goal-directed. He argued that to understand behavior, we must look at the entire sequence of actions leading to a goal. For instance, high school students study hard, not just due to past reinforcement but also to achieve the goal of getting into a good college.
Tolman introduced the idea that behavior is influenced by...
E. C. Tolman's theory of purposive behavior emphasizes that much behavior is goal-directed. He argued that to understand behavior, we must look at the entire sequence of actions leading to a goal. For instance, high school students study hard, not just due to past reinforcement but also to achieve the goal of getting into a good college.
Tolman introduced the idea that behavior is influenced by...
720
01:24Reinforcement Schedules
278
Positive reinforcement is a powerful method for teaching new behaviors to both animals and humans. B.F. Skinner demonstrated this with his experiments using rats in a Skinner box. When a rat pressed a lever, it received a food pellet. This immediate reward encouraged the rat to repeat the behavior. This method, where a reward follows every instance of the behavior, is known as continuous reinforcement. It is highly effective for establishing new behaviors quickly.
Once a behavior is learned,...
Once a behavior is learned,...
278
01:12Observational Learning
408
Albert Bandura's observational learning, also known as imitation or modeling, occurs when a person observes and imitates another's behavior. It is a quicker process than operant conditioning. A well-known example is the Bobo doll study, where children who saw an adult acting aggressively towards the doll were more likely to act aggressively when left alone, compared to those who observed a nonaggressive adult. Many psychologists view observational learning as a form of latent learning...
408
01:23Reinforcement
486
Positive and negative reinforcement are key concepts in operant conditioning, a learning process where the consequences of a behavior affect the likelihood of that behavior being repeated.
Positive reinforcement occurs when a behavior is followed by the presentation of a rewarding stimulus, increasing the frequency of that behavior. For example:
Positive reinforcement occurs when a behavior is followed by the presentation of a rewarding stimulus, increasing the frequency of that behavior. For example:
486
01:18Hierarchy of Motor Control
4.2K
The hierarchy of motor control refers to the different levels of organization and processing involved in controlling movement in the body. These levels range from higher cortical areas involved in planning and decision-making to lower spinal cord reflexes that respond automatically to external stimuli.
4.2K
01:27Associative Learning
712
Associative learning is a fundamental concept in behavioral psychology, wherein a connection is established between two stimuli or events, leading to a learned response. This process is critical in understanding how behaviors are acquired and modified. Conditioning, the mechanism through which associations are formed, can be divided into two main types: classical conditioning and operant conditioning, each elucidating different aspects of associative learning.
Classical conditioning, also known...
Classical conditioning, also known...
712