Jove
Visualize
Contact Us

Related Experiment Videos

Effective Backpropagation Training with Variable Stepsize.

George S. Androulakis1, Michael N. Vrahatis, George D. Magoulas

  • 1Department of Mathematics, University of Patras, Greece

Neural Networks : the Official Journal of the International Neural Network Society
|January 1, 1997
PubMed
Summary

This study introduces a modified steepest descent method for backpropagation training, improving convergence speed and reducing local minima. The computationally efficient algorithm demonstrates robust performance across various problems.

Related Concept Videos

You might also read

Related Articles

Articles linked to this work by shared authors, journal, and citation graph.

Sort by
Same journal

HVPNet: A bio-inspired network for general salient and camouflaged object detection.

Neural networks : the official journal of the International Neural Network Society·2026
Same journal

MT-RevSNN: Memory and time efficient reversible framework for hierarchical spiking transformer.

Neural networks : the official journal of the International Neural Network Society·2026
Same journal

MSESTA: Multi-scale evolving spatial-temporal attention for long-term traffic prediction.

Neural networks : the official journal of the International Neural Network Society·2026
Same journal

GSsplat: Generalizable semantic Gaussian splatting for novel-view synthesis in 3D scenes.

Neural networks : the official journal of the International Neural Network Society·2026
Same journal

Hierarchical feature based dual contrastive multiview clustering.

Neural networks : the official journal of the International Neural Network Society·2026
Same journal

A survey of the integration between machine learning and artificial intelligence techniques in software-defined networking.

Neural networks : the official journal of the International Neural Network Society·2026
JoVE
x logofacebook logolinkedin logoyoutube logo
ABOUT JoVE
OverviewLeadershipBlogJoVE Help Center
AUTHORS
Publishing ProcessEditorial BoardScope & PoliciesPeer ReviewFAQSubmit
LIBRARIANS
TestimonialsSubscriptionsAccessResourcesLibrary Advisory BoardFAQ
RESEARCH
JoVE JournalMethods CollectionsJoVE Encyclopedia of ExperimentsArchive
EDUCATION
JoVE CoreJoVE BusinessJoVE Science EducationJoVE Lab ManualFaculty Resource CenterFaculty Site
Terms & Conditions of Use
Privacy Policy
Policies

Area of Science:

  • Artificial Intelligence
  • Machine Learning
  • Neural Networks

Background:

  • Backpropagation training algorithms often face challenges with variable stepsize, leading to slower training and local minima convergence.
  • Existing heuristic methods aim to improve training efficiency and generalization but can be complex.

Purpose of the Study:

  • To develop a computationally efficient backpropagation training algorithm with variable stepsize.
  • To enhance convergence properties and reduce the likelihood of converging to local minima.
  • To utilize Lipschitz constant estimates without incurring additional computational overhead.

Main Methods:

  • Implementation of a modified steepest descent method for backpropagation.
  • Incorporation of a variable stepsize mechanism into the training algorithm.

Related Experiment Videos

  • Estimation of the Lipschitz constant to guide stepsize adjustments.

    • Main Results:

    • The modified algorithm demonstrated satisfactory performance across multiple test problems.
    • Numerical evidence indicated robustness and good average performance on diverse problem classes.
    • The method proved computationally efficient, offering improved convergence properties.

    Conclusions:

    • The proposed variable stepsize backpropagation algorithm is effective and efficient.
    • The approach offers a robust solution for improving neural network training.
    • Lipschitz constant estimation provides a valuable, low-cost enhancement for backpropagation.