Jove
Visualize
Contact Us
JoVE
x logofacebook logolinkedin logoyoutube logo
ABOUT JoVE
OverviewLeadershipBlogJoVE Help Center
AUTHORS
Publishing ProcessEditorial BoardScope & PoliciesPeer ReviewFAQSubmit
LIBRARIANS
TestimonialsSubscriptionsAccessResourcesLibrary Advisory BoardFAQ
RESEARCH
JoVE JournalMethods CollectionsJoVE Encyclopedia of ExperimentsArchive
EDUCATION
JoVE CoreJoVE BusinessJoVE Science EducationJoVE Lab ManualFaculty Resource CenterFaculty Site
Terms & Conditions of Use
Privacy Policy
Policies

Related Concept Videos

Uncertainty: Overview00:59

Uncertainty: Overview

In analytical chemistry, we often perform repetitive measurements to detect and minimize inaccuracies caused by both determinate and indeterminate errors. Despite the cares we take, the presence of random errors means that repeated measurements almost never have exactly the same magnitude. The collective difference between these measurements - observed values - and the estimated or expected value is called uncertainty. Uncertainty is conventionally written after the estimated or expected value.
Propagation of Uncertainty from Random Error00:59

Propagation of Uncertainty from Random Error

An experiment often consists of more than a single step. In this case, measurements at each step give rise to uncertainty. Because the measurements occur in successive steps, the uncertainty in one step necessarily contributes to that in the subsequent step. As we perform statistical analysis on these types of experiments, we must learn to account for the propagation of uncertainty from one step to the next. The propagation of uncertainty depends on the type of arithmetic operation performed on...
Reinforcement01:23

Reinforcement

Positive and negative reinforcement are key concepts in operant conditioning, a learning process where the consequences of a behavior affect the likelihood of that behavior being repeated.
Positive reinforcement occurs when a behavior is followed by the presentation of a rewarding stimulus, increasing the frequency of that behavior. For example:
Propagation of Uncertainty from Systematic Error01:10

Propagation of Uncertainty from Systematic Error

The atomic mass of an element varies due to the relative ratio of its isotopes. A sample's relative proportion of oxygen isotopes influences its average atomic mass. For instance, if we were to measure the atomic mass of oxygen from a sample, the mass would be a weighted average of the isotopic masses of oxygen in that sample. Since a single sample is not likely to perfectly reflect the true atomic mass of oxygen for all the molecules of oxygen on Earth, the mass we obtain from this particular...
State Space Representation01:27

State Space Representation

The frequency-domain technique, commonly used in analyzing and designing feedback control systems, is effective for linear, time-invariant systems. However, it falls short when dealing with nonlinear, time-varying, and multiple-input multiple-output systems. The time-domain or state-space approach addresses these limitations by utilizing state variables to construct simultaneous, first-order differential equations, known as state equations, for an nth-order system.
Consider an RLC circuit, a...
Avoidance Learning and Learned Helplessness01:14

Avoidance Learning and Learned Helplessness

Avoidance learning and learned helplessness are critical concepts in understanding behavioral responses to negative stimuli.
Avoidance learning occurs when an organism learns that a specific behavior can prevent an unpleasant outcome. For example, a student who receives a bad grade may start studying harder to avoid future poor grades. This behavior persists even when the negative outcome is no longer present. Avoidance learning is powerful because it maintains behavior in the absence of the...

You might also read

Related Articles

Articles linked to this work by shared authors, journal, and citation graph.

Sort by
Same author

Dithering suppresses half-harmonic neural synchronisation to photic stimulation in humans.

Brain stimulation·2026
Same author

Predictive Coding Model Detects Novelty on Different Levels of Representation Hierarchy.

Neural computation·2025
Same author

Dopamine encodes deep network teaching signals for individual learning trajectories.

Cell·2025
Same author

Response of Neuronal Populations to Phase-Locked Stimulation: Model-Based Predictions and Validation.

The Journal of neuroscience : the official journal of the Society for Neuroscience·2025
Same author

Reward Bases: A simple mechanism for adaptive acquisition of multiple reward types.

PLoS computational biology·2024
Same author

Computational and Neural Evidence for Altered Fast and Slow Learning from Losses in Problem Gambling.

The Journal of neuroscience : the official journal of the Society for Neuroscience·2024

Related Experiment Videos

Posterior weighted reinforcement learning with state uncertainty.

Tobias Larsen1, David S Leslie, Edmund J Collins

  • 1Department of Computer Science, University of Bristol, Bristol, UK. larsent@tcd.ie

Neural Computation
|January 27, 2010
PubMed
Summary
This summary is machine-generated.

This study introduces posterior weighted reinforcement learning to handle noisy environments. The new method updates state probabilities based on observed rewards, ensuring accurate value estimates in reinforcement learning.

Related Experiment Videos

Area of Science:

  • Computational Neuroscience
  • Machine Learning
  • Artificial Intelligence

Background:

  • Reinforcement learning (RL) typically assumes unambiguous state identification from stimuli.
  • Real-world environments present noisy stimuli, leading to state uncertainty.
  • This uncertainty complicates reward allocation and value estimation in standard RL.

Purpose of the Study:

  • To address the challenge of state uncertainty in reinforcement learning.
  • To develop a method that correctly incorporates noisy state information into RL.
  • To improve the accuracy of value estimates when environmental states are ambiguous.

Main Methods:

  • Introduced posterior weighted reinforcement learning (PWRL).
  • PWRL updates state probabilities based on observed rewards.
  • Algorithm analyzed analytically and confirmed with numerical experiments.

Main Results:

  • Ignoring state uncertainty leads to incorrect value estimates.
  • PWRL demonstrates convergence to correct reward estimates.
  • The algorithm is a variant of the expectation-maximization algorithm.

Conclusions:

  • Posterior weighted reinforcement learning effectively handles state uncertainty.
  • The method provides a rigorous approach to RL in noisy environments.
  • A neural implementation in cortico-basal-ganglia-thalamic networks is proposed.