Jove
Visualize
Contact Us
JoVE
x logofacebook logolinkedin logoyoutube logo
ABOUT JoVE
OverviewLeadershipBlogJoVE Help Center
AUTHORS
Publishing ProcessEditorial BoardScope & PoliciesPeer ReviewFAQSubmit
LIBRARIANS
TestimonialsSubscriptionsAccessResourcesLibrary Advisory BoardFAQ
RESEARCH
JoVE JournalMethods CollectionsJoVE Encyclopedia of ExperimentsArchive
EDUCATION
JoVE CoreJoVE BusinessJoVE Science EducationJoVE Lab ManualFaculty Resource CenterFaculty Site
Terms & Conditions of Use
Privacy Policy
Policies

Related Concept Videos

Time-Domain Interpretation of PD Control01:07

Time-Domain Interpretation of PD Control

83
Proportional-Derivative (PD) control is a widely used control method in various engineering systems to enhance stability and performance. In a system with only proportional control, common issues include high maximum overshoot and oscillation, observed in both the error signal and its rate of change. This behavior can be divided into three distinct phases: initial overshoot, subsequent undershoot, and gradual stabilization.
Consider the example of control of motor torque. Initially, a positive...
83
Mechanistic Models: Compartment Models in Algorithms for Numerical Problem Solving01:29

Mechanistic Models: Compartment Models in Algorithms for Numerical Problem Solving

40
Mechanistic models play a crucial role in algorithms for numerical problem-solving, particularly in nonlinear mixed effects modeling (NMEM). These models aim to minimize specific objective functions by evaluating various parameter estimates, leading to the development of systematic algorithms. In some cases, linearization techniques approximate the model using linear equations.
In individual population analyses, different algorithms are employed, such as Cauchy's method, which uses a...
40
Routh-Hurwitz Criterion II01:19

Routh-Hurwitz Criterion II

182
In the application of the Routh-Hurwitz criterion, two specific scenarios can arise that complicate stability analysis.
The first scenario occurs when a singular zero appears in the first column of the Routh table. This situation creates a division by zero issues. To resolve this, a small positive or negative number, denoted as epsilon (∈), is substituted for the zero. The stability analysis proceeds by assuming a sign for ∈. If ∈ is positive, any sign change in the first...
182
Region of Convergence01:17

Region of Convergence

377
The z-transform is a powerful mathematical tool used in the analysis of discrete-time signals and systems. It is a crucial tool in the analysis of discrete-time systems, but its convergence is limited to specific values of the complex variable z. This range of values, known as the Region of Convergence (ROC), is fundamental in determining the behavior and stability of a system or signal. The ROC defines the region in the complex plane where the z-transform converges, which can take various...
377
Region of Convergence of Laplace Tarnsform01:20

Region of Convergence of Laplace Tarnsform

484
The Region of Convergence (ROC) is a fundamental concept in signal processing and system analysis, particularly associated with the Laplace transform. The ROC represents an area in the complex plane where the Laplace transform of a given signal converges, determining the transform's applicability and utility.
Consider a decaying exponential signal that begins at a specific time. When deriving its Laplace transform, the time-domain variable is replaced with a complex variable. This...
484
Propagation of Uncertainty from Random Error00:59

Propagation of Uncertainty from Random Error

645
An experiment often consists of more than a single step. In this case, measurements at each step give rise to uncertainty. Because the measurements occur in successive steps, the uncertainty in one step necessarily contributes to that in the subsequent step. As we perform statistical analysis on these types of experiments, we must learn to account for the propagation of uncertainty from one step to the next. The propagation of uncertainty depends on the type of arithmetic operation performed on...
645

You might also read

Related Articles

Articles linked to this work by shared authors, journal, and citation graph.

Sort by
Same author

Unconventional Zero-Field-Cooling Exchange Bias in 2D Van der Waals Magnetic Heterostructures.

Nano letters·2026
Same author

A group of TCP transcription factors is a missing link in strigolactone signaling.

Journal of integrative plant biology·2026
Same author

Quaternion-based CNN for heart rate prediction from PPG.

Neural networks : the official journal of the International Neural Network Society·2026
Same author

[Patient and family predictors of group intervention outcomes in children with attention-deficit/hyperactivity disorder: a multidimensional model analysis].

Nan fang yi ke da xue xue bao = Journal of Southern Medical University·2026
Same author

A hybrid adaptive preconditioned gradient method with momentum for deep learning.

Neural networks : the official journal of the International Neural Network Society·2026
Same author

High-Performance and Cost-Effective Zwitterionic Membrane for Enhanced Electrocatalytic CO<sub>2</sub> Reduction.

Small (Weinheim an der Bergstrasse, Germany)·2026
Same journal

Adaptive memristor-based LIF neuron circuit for energy efficient SNN crossbar array.

Cognitive neurodynamics·2026
Same journal

Dynamic bi-domain discriminator adversarial network for EEG emotion recognition.

Cognitive neurodynamics·2026
Same journal

Olfactory Perception and Neural Rhythms: A Simulation-Based EEG Analysis Using Power Spectral Density FeaturesOlfactory perception and neural rhythms: a simulation-based eeg analysis using power spectral density features.

Cognitive neurodynamics·2026
Same journal

An event-related potentials account of brain predictive coding.

Cognitive neurodynamics·2026
Same journal

A recurrent neural network model for a decision-making task based on sequential evidence accumulation.

Cognitive neurodynamics·2026
Same journal

Synaptic neurotransmitter concentration modulation during learning in bio-inspired spiking neural network.

Cognitive neurodynamics·2026
See all related articles

Related Experiment Video

Updated: Jun 4, 2025

Design and Application of a Fault Detection Method Based on Adaptive Filters and Rotational Speed Estimation for an Electro-Hydrostatic Actuator
06:45

Design and Application of a Fault Detection Method Based on Adaptive Filters and Rotational Speed Estimation for an Electro-Hydrostatic Actuator

Published on: October 28, 2022

1.6K

On hyper-parameter selection for guaranteed convergence of RMSProp.

Jinlan Liu1, Dongpo Xu1, Huisheng Zhang2

  • 1School of Mathematics and Statistics, Northeast Normal University, Changchun, 130024 China.

Cognitive Neurodynamics
|December 23, 2024
PubMed
Summary
This summary is machine-generated.

A new time-varying RMSProp algorithm addresses convergence issues in deep learning. This optimization method, with a dynamic hyperparameter, ensures convergence for various objectives and improves performance on benchmark datasets.

Keywords:
ConvergenceDeep learningNeural networksNon-convex optimizationRMSProp

More Related Videos

An Experimental Platform to Study the Closed-loop Performance of Brain-machine Interfaces
10:51

An Experimental Platform to Study the Closed-loop Performance of Brain-machine Interfaces

Published on: March 10, 2011

13.7K
Measuring Attention and Visual Processing Speed by Model-based Analysis of Temporal-order Judgments
13:00

Measuring Attention and Visual Processing Speed by Model-based Analysis of Temporal-order Judgments

Published on: January 23, 2017

9.8K

Related Experiment Videos

Last Updated: Jun 4, 2025

Design and Application of a Fault Detection Method Based on Adaptive Filters and Rotational Speed Estimation for an Electro-Hydrostatic Actuator
06:45

Design and Application of a Fault Detection Method Based on Adaptive Filters and Rotational Speed Estimation for an Electro-Hydrostatic Actuator

Published on: October 28, 2022

1.6K
An Experimental Platform to Study the Closed-loop Performance of Brain-machine Interfaces
10:51

An Experimental Platform to Study the Closed-loop Performance of Brain-machine Interfaces

Published on: March 10, 2011

13.7K
Measuring Attention and Visual Processing Speed by Model-based Analysis of Temporal-order Judgments
13:00

Measuring Attention and Visual Processing Speed by Model-based Analysis of Temporal-order Judgments

Published on: January 23, 2017

9.8K

Area of Science:

  • Deep Learning Optimization
  • Stochastic Gradient Descent Variants

Background:

  • Root Mean Square Propagation (RMSProp) is a widely used optimizer in deep learning.
  • Recent studies indicate RMSProp may fail to converge to optimal solutions, even in convex scenarios.

Purpose of the Study:

  • To propose a modified RMSProp algorithm with a time-varying hyperparameter to resolve non-convergence issues.
  • To theoretically analyze the convergence properties of the proposed method for both convex and non-convex problems.

Main Methods:

  • Introducing a time-varying sequence for the hyperparameter instead of a fixed value.
  • Providing a rigorous mathematical proof for convergence to critical points in non-convex settings with a specific convergence rate.

Main Results:

  • The proposed time-varying RMSProp demonstrates convergence to critical points for smooth, non-convex objectives.
  • Theoretical convergence rate of order is established.
  • Numerical experiments confirm the advantages of time-varying RMSProp over standard RMSProp on benchmark datasets.

Conclusions:

  • The time-varying RMSProp effectively addresses the divergence issues of the standard algorithm.
  • The modified optimizer offers improved performance and theoretical guarantees for deep learning applications.
  • This work provides a deeper understanding of RMSProp's convergence behavior.