Jove
Visualize
Contact Us
JoVE
x logofacebook logolinkedin logoyoutube logo
ABOUT JoVE
OverviewLeadershipBlogJoVE Help Center
AUTHORS
Publishing ProcessEditorial BoardScope & PoliciesPeer ReviewFAQSubmit
LIBRARIANS
TestimonialsSubscriptionsAccessResourcesLibrary Advisory BoardFAQ
RESEARCH
JoVE JournalMethods CollectionsJoVE Encyclopedia of ExperimentsArchive
EDUCATION
JoVE CoreJoVE BusinessJoVE Science EducationJoVE Lab ManualFaculty Resource CenterFaculty Site
Terms & Conditions of Use
Privacy Policy
Policies

Related Concept Videos

Decision Making: P-value Method01:09

Decision Making: P-value Method

7.2K
The process of hypothesis testing based on the P-value method includes calculating the P- value using the sample data and interpreting it.
First, a specific claim about the population parameter is proposed. The claim is based on the research question and is stated in a simple form. Further, an opposing statement to the claim  is also stated. These statements can act as null and alternative hypotheses:  a null hypothesis would be a neutral statement while the alternative hypothesis can...
7.2K
Mechanistic Models: Compartment Models in Algorithms for Numerical Problem Solving01:29

Mechanistic Models: Compartment Models in Algorithms for Numerical Problem Solving

397
Mechanistic models play a crucial role in algorithms for numerical problem-solving, particularly in nonlinear mixed effects modeling (NMEM). These models aim to minimize specific objective functions by evaluating various parameter estimates, leading to the development of systematic algorithms. In some cases, linearization techniques approximate the model using linear equations.
In individual population analyses, different algorithms are employed, such as Cauchy's method, which uses a...
397
PD Controller: Design01:26

PD Controller: Design

711
In automotive engineering, car suspension systems often employ Proportional Derivative (PD) controllers to enhance performance. PD controllers are utilized to adjust the damping force in response to road conditions. A controller, acting as an amplifier with a constant gain, demonstrates proportional control, with output directly mirroring input.
Designing a continuous-data controller requires selecting and linking components like adders and integrators, which are fundamental in Proportional,...
711
Woodward–Hoffmann Selection Rules and Microscopic Reversibility01:34

Woodward–Hoffmann Selection Rules and Microscopic Reversibility

4.2K
Electrocyclic reactions, cycloadditions, and sigmatropic rearrangements are concerted pericyclic reactions that proceed via a cyclic transition state. These reactions are stereospecific and regioselective. The stereochemistry of the products depends on the symmetry characteristics of the interacting orbitals and the reaction conditions. Accordingly, pericyclic reactions are classified as either symmetry-allowed or symmetry-forbidden. Woodward and Hoffmann presented the selection criteria for...
4.2K
State Function, Exact and Inexact Differentials01:27

State Function, Exact and Inexact Differentials

105
A state function is a thermodynamic property that depends solely on the current state of a system, irrespective of its history or how it arrived at that state. These functions are represented by capital letters, such as U, H, and S, which stand for internal energy, enthalpy, and entropy, respectively.For instance, the value of internal energy depends on the system's state variables and remains unaffected by the process path. This means that whether the system underwent a linear process or a...
105
Propagation of Action Potentials01:23

Propagation of Action Potentials

13.8K
The propagation of an action potential refers to the process by which a nerve impulse, or "action potential," travels along a neuron.
Neurons (nerve cells) have a resting membrane potential, with a slightly negative charge inside compared to outside. This is maintained by ion channels, such as sodium (Na+) and potassium (K+) channels, which control the flow of ions. When a stimulus, like a touch or a signal from another neuron, triggers the neuron, sodium channels open, allowing sodium ions to...
13.8K

You might also read

Related Articles

Articles linked to this work by shared authors, journal, and citation graph.

Sort by
Same author

Early Stopping Without Validation Data in Weakly Supervised Learning.

IEEE transactions on pattern analysis and machine intelligence·2026
Same author

Instance-dependent Early Stopping for Adaptive Data Pruning.

IEEE transactions on pattern analysis and machine intelligence·2026
Same author

Class-Distribution-Aware Pseudo-Labeling for Semi-Supervised Multi-Label Learning.

IEEE transactions on pattern analysis and machine intelligence·2026
Same author

Impact of Noisy Supervision in Foundation Model Learning.

IEEE transactions on pattern analysis and machine intelligence·2025
Same author

A Fast Algorithm for the Real-Valued Combinatorial Pure Exploration of the Multi-Armed Bandit.

Neural computation·2024
Same author

Learning explainable task-relevant state representation for model-free deep reinforcement learning.

Neural networks : the official journal of the International Neural Network Society·2024

Related Experiment Videos

An Online Policy Gradient Algorithm for Markov Decision Processes with Continuous States and Actions.

Yao Ma1, Tingting Zhao2, Kohei Hatano3

  • 1Tokyo Institute of Technology, Meguro, Tokyo 152-8552, Japan mycw45@gmail.com.

Neural Computation
|January 7, 2016
PubMed
Summary

This study introduces an online policy gradient algorithm for Markov decision processes (MDPs) to minimize regret in time-varying environments. The method achieves sublinear regret bounds, offering a novel solution for continuous learning problems.

Related Experiment Videos

Area of Science:

  • Artificial Intelligence
  • Machine Learning
  • Reinforcement Learning

Background:

  • Online Markov decision processes (MDPs) present challenges due to time-varying reward functions.
  • Minimizing regret, the difference from the optimal fixed policy, is a key objective in online learning.

Purpose of the Study:

  • To develop and analyze an online policy gradient algorithm for learning time-dependent decision-making policies in MDPs.
  • To establish theoretical guarantees for the algorithm's performance in terms of regret minimization.

Main Methods:

  • An online policy gradient algorithm is proposed to address the learning problem in online MDPs.
  • Theoretical analysis is conducted to derive regret bounds under different concavity assumptions.

Main Results:

  • The algorithm achieves a regret of O(√T) under a general concavity assumption and O(log T) under strong concavity.
  • This represents the first online MDP algorithm with guarantees for continuous state, action, and parameter spaces.

Conclusions:

  • The proposed online policy gradient algorithm effectively minimizes regret in time-varying MDPs.
  • The algorithm offers a robust solution for complex, continuous learning environments, validated by experimental results.