Jove
Visualize
Contact Us

Related Concept Videos

Mechanistic Models: Compartment Models in Algorithms for Numerical Problem Solving01:29

Mechanistic Models: Compartment Models in Algorithms for Numerical Problem Solving

38
Mechanistic models play a crucial role in algorithms for numerical problem-solving, particularly in nonlinear mixed effects modeling (NMEM). These models aim to minimize specific objective functions by evaluating various parameter estimates, leading to the development of systematic algorithms. In some cases, linearization techniques approximate the model using linear equations.
In individual population analyses, different algorithms are employed, such as Cauchy's method, which uses a...
38
Linear Approximation in Time Domain01:21

Linear Approximation in Time Domain

59
Nonlinear systems often require sophisticated approaches for accurate modeling and analysis, with state-space representation being particularly effective. This method is especially useful for systems where variables and parameters vary with time or operating conditions, such as in a simple pendulum or a translational mechanical system with nonlinear springs.
For a simple pendulum with a mass evenly distributed along its length and the center of mass located at half the pendulum's length,...
59
Feedback control systems01:26

Feedback control systems

275
Feedback control systems are categorized in various ways based on their design, analysis, and signal types.
Linear feedback systems are theoretical models that simplify analysis and design. These systems operate under the principle that their output is directly proportional to their input within certain ranges. For instance, an amplifier in a control system behaves linearly as long as the input signal remains within a specific range. However, most physical systems exhibit inherent nonlinearity...
275
Time-Domain Interpretation of PD Control01:07

Time-Domain Interpretation of PD Control

79
Proportional-Derivative (PD) control is a widely used control method in various engineering systems to enhance stability and performance. In a system with only proportional control, common issues include high maximum overshoot and oscillation, observed in both the error signal and its rate of change. This behavior can be divided into three distinct phases: initial overshoot, subsequent undershoot, and gradual stabilization.
Consider the example of control of motor torque. Initially, a positive...
79
Multimachine Stability01:25

Multimachine Stability

134
Multimachine stability analysis is crucial for understanding the dynamics and stability of power systems with multiple synchronous machines. The objective is to solve the swing equations for a network of M machines connected to an N-bus power system.
In analyzing the system, the nodal equations represent the relationship between bus voltages, machine voltages, and machine currents. The nodal equation is given by:
134
Bernoulli's Equation: Problem Solving01:16

Bernoulli's Equation: Problem Solving

740
A Venturi meter is essential for measuring fluid flow rates in pipelines. It utilizes the relationship between fluid velocity and pressure described by Bernoulli's equation. When installed in a sewage system, the Venturi meter accurately determines the wastewater flow rate by measuring pressure differences.
The first step is to compute the cross-sectional areas of the pipe and the Venturi throat to analyze the pressure difference indicated by the pressure gauge. Next, the continuity...
740

You might also read

Related Articles

Articles linked to this work by shared authors, journal, and citation graph.

Sort by
Same author

Gradient Descent Provably Escapes Saddle Points in the Training of Shallow ReLU Networks.

Journal of optimization theory and applications·2024
Same author

Overcoming the curse of dimensionality in the numerical approximation of high-dimensional semilinear elliptic partial differential equations.

SN partial differential equations and applications·2024
Same author

Overcoming the curse of dimensionality in the numerical approximation of semilinear parabolic partial differential equations.

Proceedings. Mathematical, physical, and engineering sciences·2021
See all related articles
JoVE
x logofacebook logolinkedin logoyoutube logo
ABOUT JoVE
OverviewLeadershipBlogJoVE Help Center
AUTHORS
Publishing ProcessEditorial BoardScope & PoliciesPeer ReviewFAQSubmit
LIBRARIANS
TestimonialsSubscriptionsAccessResourcesLibrary Advisory BoardFAQ
RESEARCH
JoVE JournalMethods CollectionsJoVE Encyclopedia of ExperimentsArchive
EDUCATION
JoVE CoreJoVE BusinessJoVE Science EducationJoVE Lab ManualFaculty Resource CenterFaculty Site
Terms & Conditions of Use
Privacy Policy
Policies

Related Experiment Video

Updated: May 29, 2025

WheelCon: A Wheel Control-Based Gaming Platform for Studying Human Sensorimotor Control
08:18

WheelCon: A Wheel Control-Based Gaming Platform for Studying Human Sensorimotor Control

Published on: August 15, 2020

4.9K

Nonlinear Monte Carlo Methods with Polynomial Runtime for Bellman Equations of Discrete Time High-Dimensional

Christian Beck1, Arnulf Jentzen2,3, Konrad Kleinberg4

  • 1Department of Mathematics, ETH Zurich, Zurich, Switzerland.

Applied Mathematics and Optimization
|February 7, 2025
PubMed
Summary

This study introduces novel nonlinear Monte Carlo methods for approximating solutions to Bellman equations in Markov decision processes (MDPs). These methods effectively overcome the curse of dimensionality in stochastic optimal control problems.

Keywords:
Bellman equationsMarkov decision processesMonte Carlo methodsMultilevel fixed-point approximationsOptimal stopping

More Related Videos

Gain-compensation Methodology for a Sinusoidal Scan of a Galvanometer Mirror in Proportional-Integral-Differential Control Using Pre-emphasis Techniques
09:01

Gain-compensation Methodology for a Sinusoidal Scan of a Galvanometer Mirror in Proportional-Integral-Differential Control Using Pre-emphasis Techniques

Published on: April 4, 2017

8.6K
Design and Application of a Fault Detection Method Based on Adaptive Filters and Rotational Speed Estimation for an Electro-Hydrostatic Actuator
06:45

Design and Application of a Fault Detection Method Based on Adaptive Filters and Rotational Speed Estimation for an Electro-Hydrostatic Actuator

Published on: October 28, 2022

1.6K

Related Experiment Videos

Last Updated: May 29, 2025

WheelCon: A Wheel Control-Based Gaming Platform for Studying Human Sensorimotor Control
08:18

WheelCon: A Wheel Control-Based Gaming Platform for Studying Human Sensorimotor Control

Published on: August 15, 2020

4.9K
Gain-compensation Methodology for a Sinusoidal Scan of a Galvanometer Mirror in Proportional-Integral-Differential Control Using Pre-emphasis Techniques
09:01

Gain-compensation Methodology for a Sinusoidal Scan of a Galvanometer Mirror in Proportional-Integral-Differential Control Using Pre-emphasis Techniques

Published on: April 4, 2017

8.6K
Design and Application of a Fault Detection Method Based on Adaptive Filters and Rotational Speed Estimation for an Electro-Hydrostatic Actuator
06:45

Design and Application of a Fault Detection Method Based on Adaptive Filters and Rotational Speed Estimation for an Electro-Hydrostatic Actuator

Published on: October 28, 2022

1.6K

Area of Science:

  • Computational Mathematics
  • Artificial Intelligence
  • Operations Research

Background:

  • Stochastic optimal control and Markov decision processes (MDPs) are foundational for sequential decision-making under uncertainty and reinforcement learning.
  • Numerical approximation of infinite-horizon MDPs with general state spaces is crucial for solving complex control problems.
  • Bellman equations are central to characterizing value functions and optimal strategies in MDPs.

Purpose of the Study:

  • To develop and analyze numerical methods for approximating solutions to infinite-horizon MDPs with general state spaces.
  • To address the challenge of the curse of dimensionality in solving Bellman equations for stochastic optimal control.
  • To investigate the application of novel nonlinear Monte Carlo methods inspired by Q-learning and multilevel Picard approximations.

Main Methods:

  • Combines the full-history recursive multilevel Picard approximation method with Q-learning principles.
  • Introduces a class of nonlinear Monte Carlo methods for solving Bellman equations.
  • Focuses on discrete-time Markov processes and infinite-horizon optimal stopping problems.

Main Results:

  • The proposed nonlinear Monte Carlo methods effectively approximate solutions to Bellman equations.
  • Demonstrates that these methods do not suffer from the curse of dimensionality.
  • Provides a robust framework for numerical approximation in discrete-time stochastic optimal control.

Conclusions:

  • The developed nonlinear Monte Carlo methods offer a computationally efficient approach to solving complex MDPs.
  • These findings advance the numerical treatment of stochastic optimal control and reinforcement learning problems.
  • The methods are particularly suitable for problems with infinite horizons and general state spaces.