Jove
Visualize
Contact Us
JoVE
x logofacebook logolinkedin logoyoutube logo
ABOUT JoVE
OverviewLeadershipBlogJoVE Help Center
AUTHORS
Publishing ProcessEditorial BoardScope & PoliciesPeer ReviewFAQSubmit
LIBRARIANS
TestimonialsSubscriptionsAccessResourcesLibrary Advisory BoardFAQ
RESEARCH
JoVE JournalMethods CollectionsJoVE Encyclopedia of ExperimentsArchive
EDUCATION
JoVE CoreJoVE BusinessJoVE Science EducationJoVE Lab ManualFaculty Resource CenterFaculty Site
Terms & Conditions of Use
Privacy Policy
Policies

Related Experiment Video

Updated: Apr 30, 2026

Measuring the Subjective Value of Risky and Ambiguous Options using Experimental Economics and Functional MRI Methods
13:04

Measuring the Subjective Value of Risky and Ambiguous Options using Experimental Economics and Functional MRI Methods

Published on: September 19, 2012

11.6K

Algorithmic survey of parametric value function approximation.

Matthieu Geist, Olivier Pietquin

    IEEE Transactions on Neural Networks and Learning Systems
    |May 9, 2014
    PubMed
    Summary
    This summary is machine-generated.

    Related Concept Videos

    Approximate Integration01:24

    Approximate Integration

    194
    In many practical and theoretical contexts, the exact value of a definite integral may be inaccessible. This limitation typically arises when the antiderivative of a function is either unknown or cannot be expressed in a closed mathematical form. Alternatively, it can occur when a function is defined not by a formula but by a finite set of empirical data points, such as those collected during experiments. In these cases, approximate integration techniques provide a valuable solution.One of the...
    194
    Average Value of a Function01:17

    Average Value of a Function

    176
    The average value of a function over a closed interval can be interpreted geometrically as the height of a rectangle whose area equals the net area under the curve across that interval. This net area accounts for both positive and negative contributions of the function, providing a single representative value that reflects the function’s overall behaviorA practical illustration of this idea arises when monitoring the temperature inside a greenhouse over a twenty-four-hour period. Although...
    176
    Linearization and Approximation01:26

    Linearization and Approximation

    233
    Linearization is a mathematical technique used to approximate complex, nonlinear functions with simpler linear models in the vicinity of a chosen reference point. The method is based on the idea that, although a function may be difficult to evaluate exactly, its behavior near a specific input value can often be closely approximated by the tangent line at that point. This approach is particularly useful when small deviations from a known value are involved.Consider the square root function, for...
    233
    Application of Linearization and Approximation01:29

    Application of Linearization and Approximation

    193
    A drone flying through complex terrain often relies on more than one sensing method to estimate small changes in altitude. Along with direct measurements, air pressure provides a useful indirect indicator of vertical movement. Atmospheric pressure decreases as altitude increases, and this relationship is commonly described using an exponential model. Although accurate, converting pressure measurements into altitude values requires calculations that are too complex to perform repeatedly during...
    193
    State Function, Exact and Inexact Differentials01:27

    State Function, Exact and Inexact Differentials

    173
    A state function is a thermodynamic property that depends solely on the current state of a system, irrespective of its history or how it arrived at that state. These functions are represented by capital letters, such as U, H, and S, which stand for internal energy, enthalpy, and entropy, respectively.For instance, the value of internal energy depends on the system's state variables and remains unaffected by the process path. This means that whether the system underwent a linear process or a...
    173
    Area Problem01:26

    Area Problem

    312
    Determining the area of a region with straight edges is straightforward, as geometric formulas for rectangles, triangles, and polygons can be applied directly. However, traditional geometric methods are insufficient when a region has a curved boundary, such as the area under a function.fromThe area problem involves finding a systematic way to measure such regions. One approach to solving this problem is through approximation. Instead of attempting to compute the area exactly at the outset, the...
    312

    You might also read

    Related Articles

    Articles linked to this work by shared authors, journal, and citation graph.

    Sort by
    Same author

    AI evolution: bring biomimicry to language models.

    Bioinspiration & biomimetics·2025
    Same author

    Learning the propagation properties of rectangular metal plates for Lamb wave-based mapping.

    Ultrasonics·2022
    Same author

    Stable and Efficient Policy Evaluation.

    IEEE transactions on neural networks and learning systems·2018
    Same author

    Bridging the Gap Between Imitation Learning and Inverse Reinforcement Learning.

    IEEE transactions on neural networks and learning systems·2016
    Same author

    Optimism in Active Learning.

    Computational intelligence and neuroscience·2015
    Same author

    Adaptive black blood fast spin echo for end-systolic rest cardiac imaging.

    Magnetic resonance in medicine·2010
    Same journal

    Granular Ball-Based Noise-Resistant Fuzzy Multineighborhood Feature Selection via Label Enhancement and Feature Graph.

    IEEE transactions on neural networks and learning systems·2026
    Same journal

    Fighting Evolving Spam With ARTMAP Models: A Noise-Resilient Online Detection Framework.

    IEEE transactions on neural networks and learning systems·2026
    Same journal

    HyperSAT: Unsupervised Hypergraph Neural Networks for Weighted MaxSAT Problems.

    IEEE transactions on neural networks and learning systems·2026
    Same journal

    Negation of Basic Belief Assignment in Multisource Information Fusion on Dempster-Shafer Theory With Applications in Pattern Classification.

    IEEE transactions on neural networks and learning systems·2026
    Same journal

    Intervention Feasible Region and Driver Risk Capacity Aware Human-Machine Collaborative Safe Trajectory Planning.

    IEEE transactions on neural networks and learning systems·2026
    Same journal

    A Unified Differential Denoising Learning Framework With a Pre-Trained Model and Fuzzy Graph Networks for Drug-Drug Interaction Prediction.

    IEEE transactions on neural networks and learning systems·2026
    See all related articles

    This study surveys reinforcement learning (RL) methods for approximating value functions in large systems. It categorizes techniques into bootstrapping, residual, and projected fixed-point approaches for optimal control.

    Area of Science:

    • Machine Learning
    • Optimal Control Theory
    • Artificial Intelligence

    Background:

    • Reinforcement learning (RL) addresses optimal control by learning policies through system interaction.
    • The value function quantifies policy quality but is challenging to represent exactly in large systems.
    • Approximating the value function is a key challenge in scalable RL.

    Purpose of the Study:

    • To survey and categorize state-of-the-art methods for parametric value function approximation in reinforcement learning.
    • To provide a structured overview of techniques used when exact value function representation is infeasible.
    • To consolidate knowledge on different algorithmic approaches for value function approximation.

    Main Methods:

    • Categorization of value function approximation methods into three main groups: bootstrapping, residual, and projected fixed-point approaches.

    More Related Videos

    A Tactile Automated Passive-Finger Stimulator TAPS
    19:44

    A Tactile Automated Passive-Finger Stimulator TAPS

    Published on: June 3, 2009

    14.9K
    A Psychophysics Paradigm for the Collection and Analysis of Similarity Judgments
    08:12

    A Psychophysics Paradigm for the Collection and Analysis of Similarity Judgments

    Published on: March 1, 2022

    2.1K

    Related Experiment Videos

    Last Updated: Apr 30, 2026

    Measuring the Subjective Value of Risky and Ambiguous Options using Experimental Economics and Functional MRI Methods
    13:04

    Measuring the Subjective Value of Risky and Ambiguous Options using Experimental Economics and Functional MRI Methods

    Published on: September 19, 2012

    11.6K
    A Tactile Automated Passive-Finger Stimulator TAPS
    19:44

    A Tactile Automated Passive-Finger Stimulator TAPS

    Published on: June 3, 2009

    14.9K
    A Psychophysics Paradigm for the Collection and Analysis of Similarity Judgments
    08:12

    A Psychophysics Paradigm for the Collection and Analysis of Similarity Judgments

    Published on: March 1, 2022

    2.1K
  • Review of algorithms derived from these categories by considering associated cost functions.
  • Examination of minimization techniques, including stochastic gradient descent and recursive least-squares.

    • Main Results:

    • Identification and grouping of current leading techniques for value function approximation in RL.
    • Demonstration of how different cost functions and minimization methods lead to specific algorithms within the categories.
    • A comprehensive overview of the landscape of value function approximation in machine learning.

    Conclusions:

    • Value function approximation is crucial for applying RL to complex, large-scale systems.
    • The three identified categories provide a robust framework for understanding current RL approximation methods.
    • Further research can build upon these categorized approaches for improved optimal control strategies.