Jove
Visualize
Contact Us
JoVE
x logofacebook logolinkedin logoyoutube logo
ABOUT JoVE
OverviewLeadershipBlogJoVE Help Center
AUTHORS
Publishing ProcessEditorial BoardScope & PoliciesPeer ReviewFAQSubmit
LIBRARIANS
TestimonialsSubscriptionsAccessResourcesLibrary Advisory BoardFAQ
RESEARCH
JoVE JournalMethods CollectionsJoVE Encyclopedia of ExperimentsArchive
EDUCATION
JoVE CoreJoVE BusinessJoVE Science EducationJoVE Lab ManualFaculty Resource CenterFaculty Site
Terms & Conditions of Use
Privacy Policy
Policies

Related Experiment Videos

The early restart algorithm.

M Magdon-Ismail1, A F Atiya

  • 1Electrical Engineering Department, California Institute of Technology, Pasadena 91125, USA.

Neural Computation
|August 10, 2000
PubMed
Summary
This summary is machine-generated.

Related Concept Videos

You might also read

Related Articles

Articles linked to this work by shared authors, journal, and citation graph.

Sort by
Same author

Three-dimensional video compression using subband/wavelet transform with lower buffering requirements.

IEEE transactions on image processing : a publication of the IEEE Signal Processing Society·2008
Same author

A comparison between neural-network forecasting techniques--case study: river flow forecasting.

IEEE transactions on neural networks·2008
Same author

New results on recurrent network training: unifying the algorithms and accelerating convergence.

IEEE transactions on neural networks·2008
Same author

The equivalent martingale measure: an introduction to pricing using expectations.

IEEE transactions on neural networks·2008
Same author

Bankruptcy prediction for credit risk using neural networks: a survey and new results.

IEEE transactions on neural networks·2008
Same author

Density estimation and random variate generation using multilayer networks.

IEEE transactions on neural networks·2008
Same journal

A Model-Free Reinforcement Learning Implementation of Decision Making Under Uncertainty by Sequential Sampling.

Neural computation·2026
Same journal

DROP: Distributional and Regular Optimism and Pessimism for Reinforcement Learning.

Neural computation·2026
Same journal

Hierarchical Active Inference Using Successor Representations.

Neural computation·2026
Same journal

W-Kernel and Its Principal Space for Frequentist Evaluation of Bayesian Estimators.

Neural computation·2026
Same journal

A Hidden Markov Model-Inspired Sequence Classification Method for Hyperdimensional Computing.

Neural computation·2026
Same journal

Sparse Graphical Modeling for Electrophysiological Phase-Based Connectivity Using Circular Statistics.

Neural computation·2026
See all related articles

Restarting algorithms with unknown convergence times can improve performance. This study analyzes the restart mechanism, finding conditions and optimal times to enhance expected convergence speed, especially for problems with local minima.

Area of Science:

  • Computer Science
  • Machine Learning
  • Optimization Algorithms

Background:

  • Algorithms with random elements, like neural network training, often have unpredictable convergence times.
  • Local minima and high run-to-run variability can hinder efficient algorithm convergence.

Purpose of the Study:

  • To theoretically analyze the effectiveness of the restart mechanism for algorithms with unknown convergence times.
  • To determine conditions under which the restart strategy improves expected convergence time.
  • To derive the optimal restart time for such algorithms.

Main Methods:

  • Theoretical analysis of algorithm convergence time probability densities.
  • Mathematical derivation of conditions for restart mechanism effectiveness.
  • Formulation of optimal restart time calculation.

Related Experiment Videos

Main Results:

  • Identified conditions on convergence time probability density for which restart improves expected convergence time.
  • Derived a formula for calculating the optimal restart time.
  • Demonstrated applicability to steepest-descent algorithms and other cases.

Conclusions:

  • The restart mechanism is theoretically sound and can significantly improve convergence speed in specific scenarios.
  • Optimal restart time is dependent on the convergence time distribution of the algorithm.
  • This analysis provides a framework for optimizing algorithms prone to local minima or variable convergence.