Jove
Visualize
Contact Us
JoVE
x logofacebook logolinkedin logoyoutube logo
ABOUT JoVE
OverviewLeadershipBlogJoVE Help Center
AUTHORS
Publishing ProcessEditorial BoardScope & PoliciesPeer ReviewFAQSubmit
LIBRARIANS
TestimonialsSubscriptionsAccessResourcesLibrary Advisory BoardFAQ
RESEARCH
JoVE JournalMethods CollectionsJoVE Encyclopedia of ExperimentsArchive
EDUCATION
JoVE CoreJoVE BusinessJoVE Science EducationJoVE Lab ManualFaculty Resource CenterFaculty Site
Terms & Conditions of Use
Privacy Policy
Policies

Related Concept Videos

Linear Approximation in Time Domain01:21

Linear Approximation in Time Domain

Nonlinear systems often require sophisticated approaches for accurate modeling and analysis, with state-space representation being particularly effective. This method is especially useful for systems where variables and parameters vary with time or operating conditions, such as in a simple pendulum or a translational mechanical system with nonlinear springs.
For a simple pendulum with a mass evenly distributed along its length and the center of mass located at half the pendulum's length, the...
Differential Equations: Problem Solving01:21

Differential Equations: Problem Solving

When analyzing the motion of falling objects, it is essential to consider not only the force of gravity but also the opposing force of air resistance. A practical example involves releasing a heavy test weight during a safety check on a ship. As the weight falls from rest, gravity accelerates it downward while air resistance exerts an upward force that increases with velocity. This dynamic interplay of forces is well described by differential equations, which provide a mathematical framework...
Application of Linearization and Approximation01:29

Application of Linearization and Approximation

A drone flying through complex terrain often relies on more than one sensing method to estimate small changes in altitude. Along with direct measurements, air pressure provides a useful indirect indicator of vertical movement. Atmospheric pressure decreases as altitude increases, and this relationship is commonly described using an exponential model. Although accurate, converting pressure measurements into altitude values requires calculations that are too complex to perform repeatedly during...
Statically Indeterminate Problem Solving01:16

Statically Indeterminate Problem Solving

Statically indeterminate problems are those where statics alone can not determine the internal forces or reactions. Consider a structure comprising two cylindrical rods made of steel and brass. These rods are joined at point B and restrained by rigid supports at points A and C. Now, the reactions at points A and C and the deflection at point B are to be determined. This rod structure is classified as statically indeterminate as the structure has more supports than are necessary for maintaining...
Application of Nonlinear Inequalities01:29

Application of Nonlinear Inequalities

A nonlinear inequality describes a comparison involving an expression that curves or behaves more complexly than a straight line. These inequalities often appear in forms that include squares, products, or variables in the denominator.To solve such an inequality, one starts by rewriting it so that zero appears on one side. For example, the inequality:  can be factored as: This form makes it easier to identify the values that cause the expression to equal zero. In this case, the key values are 3...
Lagrange Multipliers: Problem Solving01:30

Lagrange Multipliers: Problem Solving

A silo with a cylindrical base, flat bottom, and hemispherical roof is a common design in agricultural and industrial storage due to its structural efficiency and ease of construction. Optimizing its dimensions to maximize storage capacity for a given amount of material—i.e., a fixed surface area—is a classic problem in applied calculus and engineering design. The key parameters are the radius r of the base and the height h of the cylindrical section.The total volume of the silo is obtained by...

You might also read

Related Articles

Articles linked to this work by shared authors, journal, and citation graph.

Sort by
Same author

Adaptive Learning Control of Uncertain Systems via Weight and Intrinsic Plasticity-Based Neural Networks.

IEEE transactions on neural networks and learning systems·2026
Same author

Prescribed-rate target tracking for time-delayed systems using output measurements.

Neural networks : the official journal of the International Neural Network Society·2026
Same author

Inverse Reinforcement Learning for Disturbed Networked Nonlinear Systems With Data Dropouts.

IEEE transactions on neural networks and learning systems·2025
Same author

Neuroadaptive Control With Enhanced Stability and Reliability.

IEEE transactions on neural networks and learning systems·2025
Same author

Inverse Reinforcement Learning for Discrete-Time Systems With Data Dropouts.

IEEE transactions on cybernetics·2025
Same author

Dynamic Analysis and Neural-Adaptive Prescribed-Time Control of the FO Memristive Magnetic-Field Electromechanical Transducer.

IEEE transactions on cybernetics·2024

Related Experiment Videos

Discrete-time nonlinear HJB solution using approximate dynamic programming: convergence proof.

Asma Al-Tamimi1, Frank L Lewis, Murad Abu-Khalaf

  • 1The Hashemite University, Zarqa 13115, Jordan. altamimi@hu.edu.jo

IEEE Transactions on Systems, Man, and Cybernetics. Part B, Cybernetics : a Publication of the IEEE Systems, Man, and Cybernetics Society
|July 18, 2008
PubMed
Summary
This summary is machine-generated.

Heuristic dynamic programming (HDP) converges to optimal control for nonlinear systems. This method uses neural networks, enabling implementation without system dynamics knowledge, particularly for linear quadratic regulators.

Related Experiment Videos

Area of Science:

  • Control theory
  • Artificial intelligence
  • Nonlinear systems

Background:

  • Heuristic dynamic programming (HDP) is an adaptive control technique.
  • Convergence proofs for HDP in general nonlinear systems are limited.
  • Implementing HDP often requires knowledge of system dynamics.

Purpose of the Study:

  • To prove the convergence of value-iteration-based HDP for general nonlinear systems.
  • To demonstrate HDP's ability to find optimal control and value functions.
  • To highlight HDP implementation without system dynamics knowledge.

Main Methods:

  • Utilized two neural networks: a critic NN for value function approximation and an action network for optimal control policy approximation.
  • Assumed exact solvability of value and action update equations at each iteration.
  • Applied the approach to discrete-time (DT) nonlinear optimal control problems.

Main Results:

  • Proven convergence of HDP to the optimal control and value function for general nonlinear systems.
  • Demonstrated that HDP can solve the Hamilton-Jacobi-Bellman equation.
  • Showcased HDP implementation without requiring internal system dynamics, especially for DT linear quadratic regulators (LQR).

Conclusions:

  • HDP offers a viable method for optimal control in nonlinear systems.
  • The use of two neural networks (critic and action) is effective for HDP implementation.
  • HDP's ability to function without system dynamics knowledge is a significant advantage, particularly in LQR applications.