Jove
Visualize
Contact Us
JoVE
x logofacebook logolinkedin logoyoutube logo
ABOUT JoVE
OverviewLeadershipBlogJoVE Help Center
AUTHORS
Publishing ProcessEditorial BoardScope & PoliciesPeer ReviewFAQSubmit
LIBRARIANS
TestimonialsSubscriptionsAccessResourcesLibrary Advisory BoardFAQ
RESEARCH
JoVE JournalMethods CollectionsJoVE Encyclopedia of ExperimentsArchive
EDUCATION
JoVE CoreJoVE BusinessJoVE Science EducationJoVE Lab ManualFaculty Resource CenterFaculty Site
Terms & Conditions of Use
Privacy Policy
Policies

Related Experiment Videos

Gradient descent learning algorithm overview: a general dynamical systems perspective.

P Baldi1

  • 1Div. of Biol., California Inst. of Technol., Pasadena, CA.

IEEE Transactions on Neural Networks
|January 1, 1995
PubMed
Summary
This summary is machine-generated.

Related Concept Videos

You might also read

Related Articles

Articles linked to this work by shared authors, journal, and citation graph.

Sort by
Same author

Precision Measurement of Neutrino Oscillation Parameters with 10 Years of Data from the NOvA Experiment.

Physical review letters·2026
Same author

Dual-Baseline Search for Active-to-Sterile Neutrino Oscillations in NOvA.

Physical review letters·2025
Same author

Search for CP-Violating Neutrino Nonstandard Interactions with the NOvA Experiment.

Physical review letters·2024
Same author

Intrinsic polarity inversion in III-nitride waveguides for efficient nonlinear interactions.

Optics express·2023
Same author

Measurement of the ν_{e}-Nucleus Charged-Current Double-Differential Cross Section at ⟨E_{ν}⟩=2.4  GeV Using NOvA.

Physical review letters·2023
Same author

Scintillation light detection in the 6-m drift-length ProtoDUNE Dual Phase liquid argon TPC.

The European physical journal. C, Particles and fields·2022
Same journal

Universal perceptron and DNA-like learning algorithm for binary neural networks: LSBF and PBF implementations.

IEEE transactions on neural networks·2013
Same journal

Guest editorial: special section on white box nonlinear prediction models.

IEEE transactions on neural networks·2011
Same journal

Data-based fault-tolerant control of high-speed trains with traction/braking notch nonlinearities and actuator failures.

IEEE transactions on neural networks·2011
Same journal

Guest editorial: special section on data-based control, modeling, and optimization.

IEEE transactions on neural networks·2011
Same journal

Neural network-based multiple robot simultaneous localization and mapping.

IEEE transactions on neural networks·2011
Same journal

Data-driven model-free adaptive control for a class of MIMO nonlinear discrete-time systems.

IEEE transactions on neural networks·2011
See all related articles

This study unifies gradient descent learning algorithms for neural networks using dynamical systems. This framework simplifies existing methods and aids in developing new trajectory learning algorithms.

Area of Science:

  • Artificial Intelligence
  • Machine Learning
  • Dynamical Systems Theory

Background:

  • Neural network training often employs diverse gradient descent algorithms.
  • Existing methods are often problem-specific (e.g., fixed point vs. trajectory learning) or model-specific (discrete vs. continuous).
  • Techniques like backpropagation, variational calculus, and adjoint methods have been used independently.

Purpose of the Study:

  • To present a unified framework for gradient descent learning algorithms in neural networks.
  • To organize and simplify diverse learning algorithms under a single theoretical umbrella.
  • To explore the derivation of novel algorithms and analyze inherent limitations.

Main Methods:

  • Utilizing a general framework based on dynamical systems.

Related Experiment Videos

  • Applying this framework to various neural network architectures (forward, recurrent) and learning problems (fixed point, trajectory).
  • Examining complexity and limitations of gradient descent.

    • Main Results:

    • A unified treatment that simplifies and organizes known gradient descent algorithms.
    • Demonstration of how diverse algorithms arise from the general dynamical systems approach.
    • Potential for deriving new learning algorithms within this framework.
    • Insights into the intrinsic complexity and limitations of gradient descent.

    Conclusions:

    • A dynamical systems approach offers a powerful, unifying perspective on neural network learning algorithms.
    • This unified view simplifies understanding and facilitates the development of new algorithms, particularly for trajectory learning.
    • The framework highlights fundamental aspects of gradient descent's capabilities and constraints.