Jove
Visualize
Contact Us
JoVE
x logofacebook logolinkedin logoyoutube logo
ABOUT JoVE
OverviewLeadershipBlogJoVE Help Center
AUTHORS
Publishing ProcessEditorial BoardScope & PoliciesPeer ReviewFAQSubmit
LIBRARIANS
TestimonialsSubscriptionsAccessResourcesLibrary Advisory BoardFAQ
RESEARCH
JoVE JournalMethods CollectionsJoVE Encyclopedia of ExperimentsArchive
EDUCATION
JoVE CoreJoVE BusinessJoVE Science EducationJoVE Lab ManualFaculty Resource CenterFaculty Site
Terms & Conditions of Use
Privacy Policy
Policies

Related Experiment Videos

On the momentum term in gradient descent learning algorithms.

Ning Qian1

  • 1Center for Neurobiology and Behavior, Columbia University, 722 W. 168th Street, New York, USA

Neural Networks : the Official Journal of the International Neural Network Society
|March 29, 2003
PubMed
Summary
This summary is machine-generated.

Related Concept Videos

You might also read

Related Articles

Articles linked to this work by shared authors, journal, and citation graph.

Sort by
Same author

Receptive-field sizes during remapping and uniform transsaccadic updating across the visual space.

bioRxiv : the preprint server for biology·2026
Same author

Desmoplastic Small Round Cell Tumor of the Ovary: A Case Report With Atypical Morphologic Features and Literature Review.

International journal of surgical pathology·2026
Same author

Development of a PCR-Cas12a-LFD visual detection system for highly sensitive and specific detection of Ralstonia sp., Phytophthora sp., Alternaria sp., and Pseudomonas sp. in tobacco.

Pest management science·2026
Same author

Primary epithelioid angiomatous nodule of the vocal cord: A case report and literature review.

Experimental and therapeutic medicine·2025
Same author

A retrospective and comprehensive analysis of excess life-years lost, mortality risk, and cause of death among severe mental illness in China.

BMC medicine·2025
Same author

In Situ Cross-Linked Chitosan/Hyaluronic Acid Hydrogel for Removing Kidney Stone Fragments after Lithotripsy Surgery.

ACS applied bio materials·2025

Momentum in connectionist learning accelerates training by acting like mass in a damped system. This optimization technique enhances convergence speed and expands stable learning rates for improved neural network performance.

Area of Science:

  • Machine Learning
  • Computational Neuroscience
  • Artificial Intelligence

Background:

  • Momentum is a common addition to connectionist learning algorithms, known to speed up training.
  • Rigorous studies on the underlying mechanisms of momentum's effectiveness are limited.

Purpose of the Study:

  • To elucidate the mechanisms behind the effectiveness of the momentum term in connectionist learning.
  • To provide a theoretical framework for understanding momentum's role in optimizing learning speed and stability.

Main Methods:

  • Analogizing the continuous-time momentum parameter to Newtonian particle dynamics in a viscous medium.
  • Analyzing system behavior near local minima as damped harmonic oscillators.
  • Deriving convergence bounds for learning-rate and momentum parameters in discrete-time simulations.

Related Experiment Videos

Main Results:

  • The momentum parameter is analogous to mass in a damped Newtonian system.
  • System dynamics near local minima resemble damped harmonic oscillators.
  • Momentum improves convergence by critically damping eigen components.
  • The momentum term broadens the range of learning rates for stable convergence.

Conclusions:

  • The momentum term significantly enhances learning speed and stability in connectionist algorithms.
  • Theoretical analysis provides a basis for understanding and optimizing momentum parameters.
  • The findings offer practical insights for tuning learning rates and momentum for efficient AI model training.