Jove
Visualize
Contact Us
JoVE
x logofacebook logolinkedin logoyoutube logo
ABOUT JoVE
OverviewLeadershipBlogJoVE Help Center
AUTHORS
Publishing ProcessEditorial BoardScope & PoliciesPeer ReviewFAQSubmit
LIBRARIANS
TestimonialsSubscriptionsAccessResourcesLibrary Advisory BoardFAQ
RESEARCH
JoVE JournalMethods CollectionsJoVE Encyclopedia of ExperimentsArchive
EDUCATION
JoVE CoreJoVE BusinessJoVE Science EducationJoVE Lab ManualFaculty Resource CenterFaculty Site
Terms & Conditions of Use
Privacy Policy
Policies

Related Concept Videos

Reinforcement Schedules01:24

Reinforcement Schedules

Positive reinforcement is a powerful method for teaching new behaviors to both animals and humans. B.F. Skinner demonstrated this with his experiments using rats in a Skinner box. When a rat pressed a lever, it received a food pellet. This immediate reward encouraged the rat to repeat the behavior. This method, where a reward follows every instance of the behavior, is known as continuous reinforcement. It is highly effective for establishing new behaviors quickly.
Once a behavior is learned,...
Reinforcement01:23

Reinforcement

Positive and negative reinforcement are key concepts in operant conditioning, a learning process where the consequences of a behavior affect the likelihood of that behavior being repeated.
Positive reinforcement occurs when a behavior is followed by the presentation of a rewarding stimulus, increasing the frequency of that behavior. For example:
Distribution Reliability and Automation01:25

Distribution Reliability and Automation

Distribution reliability in electrical power systems is critical for ensuring an uninterrupted power supply to consumers at minimal cost. According to IEEE Standard Terms, reliability is the probability that a device will function without failure over a specified time period or amount of usage. For electric power distribution, this translates to maintaining continuous power supply and addressing customer concerns over power outages. Several indices, as defined by IEEE Standard 1366-2012, are...
Observational Learning01:12

Observational Learning

Albert Bandura's observational learning, also known as imitation or modeling, occurs when a person observes and imitates another's behavior. It is a quicker process than operant conditioning. A well-known example is the Bobo doll study, where children who saw an adult acting aggressively towards the doll were more likely to act aggressively when left alone, compared to those who observed a nonaggressive adult. Many psychologists view observational learning as a form of latent learning because...
Associative Learning01:27

Associative Learning

Associative learning is a fundamental concept in behavioral psychology, wherein a connection is established between two stimuli or events, leading to a learned response. This process is critical in understanding how behaviors are acquired and modified. Conditioning, the mechanism through which associations are formed, can be divided into two main types: classical conditioning and operant conditioning, each elucidating different aspects of associative learning.
Classical conditioning, also known...
Randomized Experiments01:13

Randomized Experiments

The randomization process involves assigning study participants randomly to experimental or control groups based on their probability of being equally assigned. Randomization is meant to eliminate selection bias and balance known and unknown confounding factors so that the control group is similar to the treatment group as much as possible. A computer program and a random number generator can be used to assign participants to groups in a way that minimizes bias.
Simple randomization
Simple...

You might also read

Related Articles

Articles linked to this work by shared authors, journal, and citation graph.

Sort by
Same author

Evaluating digital health literacy interventions for adults 45+ years: a scoping review.

Health promotion international·2026
Same author

An Evaluation Methodology for Interactive Reinforcement Learning with Simulated Users.

Biomimetics (Basel, Switzerland)·2021
See all related articles

Related Experiment Video

Updated: Jun 21, 2026

Pavlovian Conditioned Approach Training in Rats
06:57

Pavlovian Conditioned Approach Training in Rats

Published on: February 4, 2016

ES-C51: Expected sarsa based C51 distributional reinforcement learning algorithm.

Rijul Tandon1, Peter Vamplew2, Cameron Foale2

  • 1UIET, Panjab University, Chandigarh, India.

Neural Networks : the Official Journal of the International Neural Network Society
|June 19, 2026
PubMed
Summary
This summary is machine-generated.

This study introduces Expected Sarsa C51 (ES-C51), a reinforcement learning algorithm that improves upon standard C51 by using an Expected Sarsa update. ES-C51 demonstrates enhanced stability and performance in various environments compared to Q-learning based C51.

Keywords:
C51Distributional reinforcement learning (DRL)Expected sarsa

Related Experiment Videos

Last Updated: Jun 21, 2026

Pavlovian Conditioned Approach Training in Rats
06:57

Pavlovian Conditioned Approach Training in Rats

Published on: February 4, 2016

Area of Science:

  • Artificial Intelligence
  • Machine Learning
  • Reinforcement Learning

Background:

  • Value-based reinforcement learning (RL) typically estimates only expected rewards.
  • Distributional Reinforcement Learning (DRL) estimates reward distributions for richer uncertainty information.
  • The C51 algorithm, a popular DRL method, uses Q-learning with potential instability issues when actions have similar expected rewards but different distributions.

Purpose of the Study:

  • To address the instability of the C51 algorithm in certain scenarios.
  • To propose a modified C51 algorithm (ES-C51) with improved stability and performance.
  • To evaluate the effectiveness of ES-C51 against a Q-learning based C51 (QL-C51) using softmax exploration.

Main Methods:

  • Modified the C51 algorithm by replacing the greedy Q-learning update with an Expected Sarsa update.
  • Implemented a softmax calculation to integrate information from all actions at a state.
  • Compared ES-C51 against QL-C51 (standard C51 with softmax exploration) on classic control environments and Atari games.

Main Results:

  • ES-C51 demonstrated reduced instability when actions had similar expected rewards.
  • The proposed ES-C51 algorithm achieved higher performance across numerous evaluated environments.
  • ES-C51 consistently outperformed QL-C51 in the conducted experiments.

Conclusions:

  • The Expected Sarsa update is a viable and effective modification for the C51 algorithm.
  • ES-C51 offers a more stable and higher-performing approach to distributional reinforcement learning.
  • This modification enhances the agent's ability to learn superior policies in complex environments.