Jove
Visualize
Contact Us
JoVE
x logofacebook logolinkedin logoyoutube logo
ABOUT JoVE
OverviewLeadershipBlogJoVE Help Center
AUTHORS
Publishing ProcessEditorial BoardScope & PoliciesPeer ReviewFAQSubmit
LIBRARIANS
TestimonialsSubscriptionsAccessResourcesLibrary Advisory BoardFAQ
RESEARCH
JoVE JournalMethods CollectionsJoVE Encyclopedia of ExperimentsArchive
EDUCATION
JoVE CoreJoVE BusinessJoVE Science EducationJoVE Lab ManualFaculty Resource CenterFaculty Site
Terms & Conditions of Use
Privacy Policy
Policies

Related Concept Videos

Linear Approximation in Time Domain01:21

Linear Approximation in Time Domain

Nonlinear systems often require sophisticated approaches for accurate modeling and analysis, with state-space representation being particularly effective. This method is especially useful for systems where variables and parameters vary with time or operating conditions, such as in a simple pendulum or a translational mechanical system with nonlinear springs.
For a simple pendulum with a mass evenly distributed along its length and the center of mass located at half the pendulum's length, the...
Distributed Loads: Problem Solving01:21

Distributed Loads: Problem Solving

Beams are structural elements commonly employed in engineering applications requiring different load-carrying capacities. The first step in analyzing a beam under a distributed load is to simplify the problem by dividing the load into smaller regions, which allows one to consider each region separately and calculate the magnitude of the equivalent resultant load acting on each portion of the beam. The magnitude of the equivalent resultant load for each region can be determined by calculating...
Application of Linearization and Approximation01:29

Application of Linearization and Approximation

A drone flying through complex terrain often relies on more than one sensing method to estimate small changes in altitude. Along with direct measurements, air pressure provides a useful indirect indicator of vertical movement. Atmospheric pressure decreases as altitude increases, and this relationship is commonly described using an exponential model. Although accurate, converting pressure measurements into altitude values requires calculations that are too complex to perform repeatedly during...
Linear Approximation in Frequency Domain01:26

Linear Approximation in Frequency Domain

Linear systems are characterized by two main properties: superposition and homogeneity. Superposition allows the response to multiple inputs to be the sum of the responses to each individual input. Homogeneity ensures that scaling an input by a scalar results in the response being scaled by the same scalar.
In contrast, nonlinear systems do not inherently possess these properties. However, for small deviations around an operating point, a nonlinear system can often be approximated as linear.
Linearization and Approximation01:26

Linearization and Approximation

Linearization is a mathematical technique used to approximate complex, nonlinear functions with simpler linear models in the vicinity of a chosen reference point. The method is based on the idea that, although a function may be difficult to evaluate exactly, its behavior near a specific input value can often be closely approximated by the tangent line at that point. This approach is particularly useful when small deviations from a known value are involved.Consider the square root function, for...
Linear Approximations01:23

Linear Approximations

For a differentiable function of two variables, linear approximation estimates values near a known point by replacing the curved surface with its tangent plane. Consider the function\begin{equation*}f(x,y)=x^2+3y^2\end{equation*}near the point (2, 1). The exact value at this point is f(2, 1) = 22 + 3(1)2 = 4 + 3 = 7.The linear approximation of f(x, y)) near (a, b) is\begin{equation*}L(x,y)=f(a,b)+f_x(a,b)(x-a)+f_y(a,b)(y-b)\end{equation*}First, compute the partial derivatives: fx(x, y) = 2x and...

You might also read

Related Articles

Articles linked to this work by shared authors, journal, and citation graph.

Sort by
Same author

Using neural networks to understand the information that guides behavior: a case study in visual navigation.

Methods in molecular biology (Clifton, N.J.)·2014
Same author

A model of ant route navigation driven by scene familiarity.

PLoS computational biology·2012
Same author

How might ants use panoramic views for route navigation?

The Journal of experimental biology·2011
Same author

Animal cognition: multi-modal interactions in ant learning.

Current biology : CB·2010

Related Experiment Videos

Reinforcement learning in continuous time and space: interference and not ill conditioning is the main problem when

Bart Baddeley1

  • 1Centre for Computational Neuroscience and Robotics, Department of Informatics, University of Sussex, Brighton, UK.

IEEE Transactions on Systems, Man, and Cybernetics. Part B, Cybernetics : a Publication of the IEEE Systems, Man, and Cybernetics Society
|July 18, 2008
PubMed
Summary

Negative interference hinders distributed function approximation in reinforcement learning (RL). A pseudopattern rehearsal strategy significantly speeds up learning in multilayer perceptron networks for RL tasks.

Related Experiment Videos

Area of Science:

  • Artificial Intelligence
  • Machine Learning
  • Robotics

Background:

  • Reinforcement learning (RL) often requires function approximation for continuous or high-dimensional problems.
  • Distributed nonlinear models are less preferred than local linear models in RL function approximation.
  • Negative interference, where new data disrupts prior learning, is a key challenge in distributed RL architectures.

Purpose of the Study:

  • To investigate the effectiveness of different learning strategies in multilayer perceptron (MLP) networks for RL value function approximation.
  • To address the problem of negative interference in distributed function approximation for RL.
  • To compare gradient descent, vario-eta, and pseudopattern rehearsal for learning in MLP networks.

Main Methods:

  • Utilized the continuous temporal difference (TD) learning algorithm TD(lambda) with an MLP network.
  • Applied three learning approaches: simple gradient descent, vario-eta, and pseudopattern rehearsal.
  • Evaluated performance on a limited-torque pendulum swing-up task.

Main Results:

  • MLP networks can approximate value functions in the pendulum task but require extensive training.
  • The vario-eta method destabilized learning, leading to convergence failure.
  • Pseudopattern rehearsal substantially accelerated the learning process.

Conclusions:

  • Negative interference poses a significant challenge in RL function approximation, more so than ill conditioning for this task.
  • Pseudopattern rehearsal is an effective strategy for mitigating negative interference and improving learning speed in RL.
  • MLP networks are viable for RL value function approximation, provided interference issues are addressed.