Related Experiment Video
Updated: Feb 1, 2026
09:50
Procedure for Human Saphenous Veins Ex Vivo Perfusion and External Reinforcement
Published on: October 1, 2014
11.8K
Offline replay supports planning in human reinforcement learning
Ida Momennejad1, A Ross Otto2, Nathaniel D Daw1
1Princeton Neuroscience Institute, Princeton University, New Jersey, United States.
Elife
|December 15, 2018
Summary
The brain uses offline replay to integrate information for planning complex decisions. Uncertainty signals, like unsigned prediction errors, trigger this replay, improving goal-directed behavior.
Area of Science:
- Neuroscience
- Cognitive Science
- Computational Neuroscience
Background:
- Sequential decision-making requires integrating past information, posing computational challenges.
- Online integration at decision time is costly; the role of offline replay in planning remains unclear.
Purpose of the Study:
- Investigate if offline replay aids planning by integrating representations.
- Determine if unsigned prediction errors (uncertainty) trigger replay for planning.
Main Methods:
- Designed a 2-step revaluation task for functional magnetic resonance imaging (fMRI).
- Measured neural activity and behavioral replanning during reward integration tasks.
Main Results:
- Off-task replay patterns predicted subsequent replanning.
- Neural sensitivity to uncertainty predicted replay and replanning.
- Hippocampal and anterior cingulate activity increased during revaluation requiring planning.
Conclusions:
- Offline replay facilitates integrating representations for informed decision-making.
- Uncertainty signals guide replay, supporting adaptive planning and goal-directed behavior.
Keywords:
Dynacognitive computational neurosciencedecision-makingfMRIhippocampushumanlearning and memorymemorymodel-based learningneuroscienceoffline memory processesplanningprediction errorprioritized replayreinforcement learningreplayrepresentation learningreward revaluationuncertaintyMore Related Videos
Related Concept Videos
01:23Reinforcement
918
Positive and negative reinforcement are key concepts in operant conditioning, a learning process where the consequences of a behavior affect the likelihood of that behavior being repeated.
Positive reinforcement occurs when a behavior is followed by the presentation of a rewarding stimulus, increasing the frequency of that behavior. For example:
Positive reinforcement occurs when a behavior is followed by the presentation of a rewarding stimulus, increasing the frequency of that behavior. For example:
918
01:27Corrosion of Reinforcement
577
The corrosion of steel reinforcement within concrete is a process influenced by the material's inherent properties and external factors. The high pH level of around 13, provided by calcium hydroxide present in concrete, initially protects the steel reinforcement by promoting the formation of a passive iron oxide layer on its surface.
However, over time and under certain conditions like carbonation, chloride ingress, and cracking this protective state can be compromised. Steel has areas with...
However, over time and under certain conditions like carbonation, chloride ingress, and cracking this protective state can be compromised. Steel has areas with...
577
01:24Reinforcement Schedules
501
Positive reinforcement is a powerful method for teaching new behaviors to both animals and humans. B.F. Skinner demonstrated this with his experiments using rats in a Skinner box. When a rat pressed a lever, it received a food pellet. This immediate reward encouraged the rat to repeat the behavior. This method, where a reward follows every instance of the behavior, is known as continuous reinforcement. It is highly effective for establishing new behaviors quickly.
Once a behavior is learned,...
Once a behavior is learned,...
501
01:28Self-Help Support Groups
352
Self-help support groups are voluntary, community-based organizations that provide a platform for individuals with shared concerns to exchange support, insights, and practical strategies for coping with life challenges. Typically led by group members or paraprofessionals, these groups form a cornerstone of mental health care, especially in reaching populations that are underserved by traditional healthcare systems.
Accessibility and Cost-Effectiveness
One of the primary strengths of self-help...
Accessibility and Cost-Effectiveness
One of the primary strengths of self-help...
352
01:25Reinforcements in Concrete
468
Reinforced concrete is a composite material used extensively in construction, combining the compressive strength of concrete with the tensile strength of steel. This synergy is essential as concrete, while excellent at resisting compression, is weak under tension. Steel bars, or rebars, are embedded in the concrete to handle these tensile forces. The choice of steel is strategic; it shares a similar coefficient of thermal expansion with concrete, which ensures uniformity in response to...
468
01:23Sampling Plans
985
Sampling is a crucial step in analytical chemistry, allowing researchers to collect representative data from a large population. Common sampling methods include random, judgmental, systematic, stratified, and cluster sampling.
Random sampling is a method where each member of the population has an equal chance of being selected for the sample. It involves selecting individuals randomly, often using random number generators or lottery-type methods. For example, when analyzing the properties of a...
Random sampling is a method where each member of the population has an equal chance of being selected for the sample. It involves selecting individuals randomly, often using random number generators or lottery-type methods. For example, when analyzing the properties of a...
985