Jove
Visualize
Contact Us
JoVE
x logofacebook logolinkedin logoyoutube logo
ABOUT JoVE
OverviewLeadershipBlogJoVE Help Center
AUTHORS
Publishing ProcessEditorial BoardScope & PoliciesPeer ReviewFAQSubmit
LIBRARIANS
TestimonialsSubscriptionsAccessResourcesLibrary Advisory BoardFAQ
RESEARCH
JoVE JournalMethods CollectionsJoVE Encyclopedia of ExperimentsArchive
EDUCATION
JoVE CoreJoVE BusinessJoVE Science EducationJoVE Lab ManualFaculty Resource CenterFaculty Site
Terms & Conditions of Use
Privacy Policy
Policies

Related Concept Videos

State Space Representation01:27

State Space Representation

593
The frequency-domain technique, commonly used in analyzing and designing feedback control systems, is effective for linear, time-invariant systems. However, it falls short when dealing with nonlinear, time-varying, and multiple-input multiple-output systems. The time-domain or state-space approach addresses these limitations by utilizing state variables to construct simultaneous, first-order differential equations, known as state equations, for an nth-order system.
Consider an RLC circuit, a...
593
Graphical Representation of Inequalities01:28

Graphical Representation of Inequalities

221
The graph of the equation where y equals x squared forms a curve known as a parabola. This curve acts as a boundary in the coordinate plane, dividing it into distinct regions based on the relative position of points.When the equality sign in the equation is replaced with an inequality—such as greater than, less than, greater than or equal to, or less than or equal to—the graphical representation changes from a single curve into a broader shaded area that signifies the set of all...
221
Control Volume and System Representations01:16

Control Volume and System Representations

1.6K
Two key frameworks are employed to analyze mass, energy, and momentum transfer: the control volume approach and the system approach. These frameworks offer different perspectives, depending on whether the focus is on a specific region in space (control volume approach) or a defined mass of fluid (system approach).
The control volume approach considers a stationary region in space through which fluid flows. This region is bounded by a control surface.  For instance, in the case of water...
1.6K
Vector Representation of Complex Numbers01:16

Vector Representation of Complex Numbers

553
Complex numbers, represented in Cartesian coordinates, can also be visualized as vectors. These vectors can be expressed in polar form, emphasizing their magnitude and angle. When a complex number is input into a function, the output is another complex number, highlighting the function's zero point from which the vector representation can originate.
Consider a function defined as the product of the complex factors in the numerator divided by the product of the complex factors in the...
553
Graphical and Analytic Representation of Sinusoids01:20

Graphical and Analytic Representation of Sinusoids

988
Analyzing two sinusoidal voltages with equal amplitude and period but different phases on an oscilloscope, an instrument used to display and analyze waveforms, involves a three-step process.
The first step is measuring the peak-to-peak value, which is twice the amplitude of the sinusoid. This provides information about the maximum voltage swing of the waveform.
Secondly, the period and angular frequency are determined. The period is the time taken for one complete cycle of the waveform, while...
988
Computed Tomography01:10

Computed Tomography

8.8K
Tomography refers to imaging by sections. Computed tomography (CT) is a non-invasive imaging technique that uses computers to analyze several cross-sectional X-rays to reveal minute details about structures in the body.
The technique was invented in the 1970s and is based on the principle that as X-rays pass through the body, they are absorbed or reflected at different levels. In the technique, a patient lies on a motorized platform while a computerized axial tomography (CAT) scanner rotates...
8.8K

You might also read

Related Articles

Articles linked to this work by shared authors, journal, and citation graph.

Sort by
Same author

Human-level learning of complex novel tasks as theory-based modelling, exploration and planning.

Philosophical transactions. Series A, Mathematical, physical, and engineering sciences·2026
Same author

Artificial intelligence for science: The easy and hard problems.

Philosophical transactions. Series A, Mathematical, physical, and engineering sciences·2026
Same author

Reconciling time and prediction error theories of associative learning.

Nature communications·2025
Same author

Quantifying the Cost of Context Sensitivity in Decision-Making.

Topics in cognitive science·2025
Same author

Bayesian estimation yields anti-Weber variability.

PNAS nexus·2025
Same author

On the Robustness and Provenance of the Gambler's Fallacy.

Psychological science·2025

Related Experiment Video

Updated: Feb 7, 2026

High-precision Electromagnetic Flowmeter with Empty Pipe Detection via Complex Programmable Logic Device-based Waveform Recognition
05:11

High-precision Electromagnetic Flowmeter with Empty Pipe Detection via Complex Programmable Logic Device-based Waveform Recognition

Published on: June 27, 2025

680

The Successor Representation: Its Computational Logic and Neural Substrates.

Samuel J Gershman1

  • 1Department of Psychology and Center for Brain Science, Harvard University, Cambridge, Massachusetts 02138 gershman@fas.harvard.edu.

The Journal of Neuroscience : the Official Journal of the Society for Neuroscience
|July 15, 2018
PubMed
Summary

The brain uses a successor representation to encode environmental states based on predictive relationships, aiding reinforcement learning. This framework balances computational efficiency and flexibility in learning.

Keywords:
cognitive mapdopaminehippocampusreinforcement learningreward

More Related Videos

In vivo Dual Substrate Bioluminescent Imaging
07:33

In vivo Dual Substrate Bioluminescent Imaging

Published on: October 11, 2011

16.1K
Conscious and Non-conscious Representations of Emotional Faces in Asperger's Syndrome
08:31

Conscious and Non-conscious Representations of Emotional Faces in Asperger's Syndrome

Published on: July 31, 2016

14.5K

Related Experiment Videos

Last Updated: Feb 7, 2026

High-precision Electromagnetic Flowmeter with Empty Pipe Detection via Complex Programmable Logic Device-based Waveform Recognition
05:11

High-precision Electromagnetic Flowmeter with Empty Pipe Detection via Complex Programmable Logic Device-based Waveform Recognition

Published on: June 27, 2025

680
In vivo Dual Substrate Bioluminescent Imaging
07:33

In vivo Dual Substrate Bioluminescent Imaging

Published on: October 11, 2011

16.1K
Conscious and Non-conscious Representations of Emotional Faces in Asperger's Syndrome
08:31

Conscious and Non-conscious Representations of Emotional Faces in Asperger's Syndrome

Published on: July 31, 2016

14.5K

Area of Science:

  • Neuroscience
  • Cognitive Science
  • Computational Neuroscience

Background:

  • Reinforcement learning algorithms in the brain are well-studied.
  • Understanding the representations of states and actions is less developed.
  • Brain's computational architecture imposes constraints on learning.

Purpose of the Study:

  • Investigate desirable representations for the brain's reinforcement learning.
  • Introduce and review the successor representation concept.
  • Provide a framework for understanding brain's reinforcement learning efficiency and flexibility.

Main Methods:

  • Review of recent behavioral and neural studies.
  • Exploration of computational studies on successor representation extensions.
  • Organizing findings within a broader theoretical framework.

Main Results:

  • The successor representation encodes states by their predictive relationships.
  • Evidence from behavioral and neural studies supports the successor representation.
  • Computational studies have extended the successor representation concept.

Conclusions:

  • The successor representation offers a valuable perspective on how the brain learns.
  • This representation helps reconcile efficiency and flexibility in reinforcement learning.
  • Further research can build upon this framework to understand neural computation.