Jove
Visualize
Contact Us
JoVE
x logofacebook logolinkedin logoyoutube logo
ABOUT JoVE
OverviewLeadershipBlogJoVE Help Center
AUTHORS
Publishing ProcessEditorial BoardScope & PoliciesPeer ReviewFAQSubmit
LIBRARIANS
TestimonialsSubscriptionsAccessResourcesLibrary Advisory BoardFAQ
RESEARCH
JoVE JournalMethods CollectionsJoVE Encyclopedia of ExperimentsArchive
EDUCATION
JoVE CoreJoVE BusinessJoVE Science EducationJoVE Lab ManualFaculty Resource CenterFaculty Site
Terms & Conditions of Use
Privacy Policy
Policies

Related Experiment Video

Updated: Jun 19, 2026

A Method for Remotely Silencing Neural Activity in Rodents During Discrete Phases of Learning
09:22

A Method for Remotely Silencing Neural Activity in Rodents During Discrete Phases of Learning

Published on: June 22, 2015

Temporal-difference reinforcement learning with distributed representations.

Zeb Kurth-Nelson1, A David Redish

  • 1Department of Neuroscience, University of Minnesota, Minneapolis, Minnesota, United States of America.

Plos One
|October 21, 2009
PubMed
Summary
This summary is machine-generated.

Related Concept Videos

Reinforcement Schedules01:24

Reinforcement Schedules

Positive reinforcement is a powerful method for teaching new behaviors to both animals and humans. B.F. Skinner demonstrated this with his experiments using rats in a Skinner box. When a rat pressed a lever, it received a food pellet. This immediate reward encouraged the rat to repeat the behavior. This method, where a reward follows every instance of the behavior, is known as continuous reinforcement. It is highly effective for establishing new behaviors quickly.
Once a behavior is learned,...
Associative Learning01:27

Associative Learning

Associative learning is a fundamental concept in behavioral psychology, wherein a connection is established between two stimuli or events, leading to a learned response. This process is critical in understanding how behaviors are acquired and modified. Conditioning, the mechanism through which associations are formed, can be divided into two main types: classical conditioning and operant conditioning, each elucidating different aspects of associative learning.
Classical conditioning, also known...
Observational Learning01:12

Observational Learning

Albert Bandura's observational learning, also known as imitation or modeling, occurs when a person observes and imitates another's behavior. It is a quicker process than operant conditioning. A well-known example is the Bobo doll study, where children who saw an adult acting aggressively towards the doll were more likely to act aggressively when left alone, compared to those who observed a nonaggressive adult. Many psychologists view observational learning as a form of latent learning because...
State Space Representation01:27

State Space Representation

The frequency-domain technique, commonly used in analyzing and designing feedback control systems, is effective for linear, time-invariant systems. However, it falls short when dealing with nonlinear, time-varying, and multiple-input multiple-output systems. The time-domain or state-space approach addresses these limitations by utilizing state variables to construct simultaneous, first-order differential equations, known as state equations, for an nth-order system.
Consider an RLC circuit, a...
Storage01:23

Storage

A schema is a mental framework that helps individuals organize and interpret information. Schemata, formed from previous experiences, influence how we process new information: how we encode it, the inferences we make, and how we retrieve it. For instance, a schema for what a typical classroom looks like might include desks, a teacher's desk, a whiteboard, and students in such an environment. This expectation helps us quickly understand and navigate new classrooms without needing to analyze each...

You might also read

Related Articles

Articles linked to this work by shared authors, journal, and citation graph.

Sort by
Same author

Coordinated Representational Drift Across the Mouse Cortex.

bioRxiv : the preprint server for biology·2026
Same author

Hippocampal representations differentiate reactive and anticipatory responses during foraging under threat.

bioRxiv : the preprint server for biology·2026
Same author

Technological <i>folie à deux</i>: feedback loops between AI chatbots and mental health.

Nature. Mental health·2026
Same author

Functional reorganization of motor cortex connectivity during learning.

bioRxiv : the preprint server for biology·2026
Same author

Reasoning with programs in replay.

bioRxiv : the preprint server for biology·2025
Same author

Individual differences in decision-making shape how mesolimbic dopamine regulates choice confidence and change-of-mind.

Nature neuroscience·2025
Same journal

Invaders taking over-Mollusc faunal change in volcanic barrier lakes of the Albertine Rift biodiversity hotspot.

PloS one·2026
Same journal

AI-driven molecular diversification and ligand-based optimization of macitentan derivatives targeting VEGFR1 and endothelin signaling pathways.

PloS one·2026
Same journal

Performance patterns and records in the world aquatics masters championships: Where do the most frequently represented nations among the top-ten masters swimmers come from?

PloS one·2026
Same journal

Modeling diurnal Temperature-Rainfall relationships under multicollinearity using PLS-SEM: A case study of Ghana.

PloS one·2026
Same journal

Organizational culture, social capital, and emergency capacity in primary healthcare institutions: A cross-sectional structural equation modeling study comparing ordinary and older communities.

PloS one·2026
Same journal

Impact of kidney function on the metabolome in the general population.

PloS one·2026
See all related articles

This study introduces a temporal-difference reinforcement learning model with distributed belief representations and discounting factors. This model explains dopamine signals and hyperbolic discounting observed in animal behavior experiments.

Area of Science:

  • Computational Neuroscience
  • Reinforcement Learning Theory
  • Behavioral Economics

Background:

  • Temporal-difference (TD) algorithms are foundational models for reinforcement learning (RL).
  • Existing TD models face challenges in representing complex state-belief distributions and discounting behaviors.
  • Understanding the neural basis of decision-making, particularly dopamine's role, remains a key research area.

Purpose of the Study:

  • To investigate distributed representations of belief and discounting factors within TD RL algorithms.
  • To develop a novel TD RL model that integrates distributed state-belief and individual discounting factors.
  • To elucidate the relationship between computational models of learning and observed neurobiological and behavioral phenomena.

Main Methods:

Related Experiment Videos

Last Updated: Jun 19, 2026

A Method for Remotely Silencing Neural Activity in Rodents During Discrete Phases of Learning
09:22

A Method for Remotely Silencing Neural Activity in Rodents During Discrete Phases of Learning

Published on: June 22, 2015

  • Development of a TD RL model incorporating "micro-Agents" with distributed state-belief and separate exponential discounting factors.
  • Analysis of the model's value-error (delta) signal in relation to dopamine recordings in conditioning paradigms.
  • Examination of the model's behavioral output, specifically hyperbolic discounting, in comparison to experimental data.

    • Main Results:

    • The model's delta signal aligns with dopamine signals observed in animal reward conditioning.
    • Distributed belief representation explains dopamine decreases in overtrained animals and differences in conditioning paradigms.
    • The integrated discounting factors lead to emergent hyperbolic discounting behavior, consistent with empirical findings.

    Conclusions:

    • The proposed TD RL model offers a unified framework for understanding distributed belief, discounting, and dopamine signaling.
    • This computational approach provides insights into the neural mechanisms underlying flexible decision-making and learning.
    • The model's success in replicating behavioral and neurobiological data highlights the importance of distributed representations in RL.