Jove
Visualize
Contact Us
JoVE
x logofacebook logolinkedin logoyoutube logo
ABOUT JoVE
OverviewLeadershipBlogJoVE Help Center
AUTHORS
Publishing ProcessEditorial BoardScope & PoliciesPeer ReviewFAQSubmit
LIBRARIANS
TestimonialsSubscriptionsAccessResourcesLibrary Advisory BoardFAQ
RESEARCH
JoVE JournalMethods CollectionsJoVE Encyclopedia of ExperimentsArchive
EDUCATION
JoVE CoreJoVE BusinessJoVE Science EducationJoVE Lab ManualFaculty Resource CenterFaculty Site
Terms & Conditions of Use
Privacy Policy
Policies

Related Concept Videos

Gradient Vectors and Their Applications01:19

Gradient Vectors and Their Applications

Every point on a topographical map corresponds to a particular elevation, so the landscape can be modeled as a surface whose height depends on horizontal position. From any given location, a hiker may face infinitely many directions, but only one direction produces the fastest possible increase in elevation. This unique route is called the direction of steepest ascent, and in multivariable calculus, it is represented by the gradient vector of the elevation function.The gradient vector points...
Significance of the Gradient Vector01:27

Significance of the Gradient Vector

A surface defined by a function of two variables can be understood by examining how it changes along specific directions. When one variable is held constant, the surface reduces to a curve that reflects variation in the other variable. For example, fixing one variable and moving parallel to a coordinate axis produces a cross-sectional curve. The slope of this curve at a given point represents how the function changes in that particular direction, providing a measure of local steepness.By...
Maximizing the Directional Derivative01:25

Maximizing the Directional Derivative

The directional derivative is a central concept in multivariable calculus that describes how a function changes at a given point when moving in a specified direction. This direction is represented by a unit vector, ensuring that only the orientation influences the rate of change. By varying the direction, different rates of change can be observed, demonstrating that the directional derivative depends strongly on the chosen direction.The directional derivative is computed using the gradient...

You might also read

Related Articles

Articles linked to this work by shared authors, journal, and citation graph.

Sort by
Same author

A 12-channel, real-time near-infrared spectroscopy instrument for brain-computer interface applications.

Annual International Conference of the IEEE Engineering in Medicine and Biology Society. IEEE Engineering in Medicine and Biology Society. Annual International Conference·2009
Same author

Blind source separation by sparse decomposition in a signal dictionary.

Neural computation·2001
Same author

Time-Skew Hebb rule in a nonisopotential neuron.

Neural computation·1995
Same author

Comments on ;Dynamic programming approach to optimal weight selection in multilayer neural networks' [with reply].

IEEE transactions on neural networks·1992

Related Experiment Videos

Gradient calculations for dynamic recurrent neural networks: a survey.

B A Pearlmutter1

  • 1Learning Syst. Dept., Siemens Corp. Res. Inc., Princeton, NJ.

IEEE Transactions on Neural Networks
|January 1, 1995
PubMed
Summary

This study unifies learning algorithms for recurrent neural networks (RNNs), comparing fixed and non-fixed point methods. It offers insights into training, simulation, and computational aspects of continuous-time RNNs.

Related Experiment Videos

Area of Science:

  • Artificial Intelligence
  • Machine Learning
  • Computational Neuroscience

Background:

  • Recurrent neural networks (RNNs) are crucial for processing sequential data.
  • Various learning algorithms exist for RNNs, often lacking a unified framework.

Purpose of the Study:

  • To present a common framework for diverse RNN learning algorithms.
  • To analyze and compare fixed-point and non-fixed-point learning techniques.
  • To discuss practical aspects of training and simulating continuous-time RNNs.

Main Methods:

  • Unified presentation of learning algorithms including recurrent backpropagation, deterministic Boltzmann machines, backpropagation through time, Elman's history cutoff, and Jordan's output feedback.
  • Discussion of forward propagation using adjoint equations.
  • Comparative analysis of temporally continuous versus clocked neural networks.

Main Results:

  • A unified framework reveals generalizations across different RNN learning algorithms.
  • Advantages and disadvantages of continuous-time RNNs are elucidated.
  • Practical training "tricks of the trade" for continuous-time and recurrent networks are provided.

Conclusions:

  • The unified framework enhances understanding and potential generalization of RNN learning.
  • Practical considerations for continuous-time RNNs are addressed, aiding implementation.
  • Analysis of computational complexity and learning speed offers guidance for algorithm selection.