Jove
Visualize
Contact Us
JoVE
x logofacebook logolinkedin logoyoutube logo
ABOUT JoVE
OverviewLeadershipBlogJoVE Help Center
AUTHORS
Publishing ProcessEditorial BoardScope & PoliciesPeer ReviewFAQSubmit
LIBRARIANS
TestimonialsSubscriptionsAccessResourcesLibrary Advisory BoardFAQ
RESEARCH
JoVE JournalMethods CollectionsJoVE Encyclopedia of ExperimentsArchive
EDUCATION
JoVE CoreJoVE BusinessJoVE Science EducationJoVE Lab ManualFaculty Resource CenterFaculty Site
Terms & Conditions of Use
Privacy Policy
Policies

Related Experiment Videos

Long short-term memory

S Hochreiter1, J Schmidhuber

  • 1Fakultät für Informatik, Technische Universität München, Germany.

Neural Computation
|October 23, 1997
PubMed
Summary
This summary is machine-generated.

Long short-term memory (LSTM) networks efficiently learn long-term dependencies by enabling constant error flow, overcoming limitations of traditional recurrent backpropagation for complex sequence tasks.

Related Concept Videos

You might also read

Related Articles

Articles linked to this work by shared authors, journal, and citation graph.

Sort by
Same author

African swine fever: a global factor affecting agricultural markets over the medium term.

Revue scientifique et technique (International Office of Epizootics)·2022
Same author

Optimising traceability in trade for live animals and animal products with digital technologies.

Revue scientifique et technique (International Office of Epizootics)·2020
Same author

Salvage therapy with everolimus reduces the severity of treatment-refractory chronic GVHD without impairing disease control: a dual center retrospective analysis.

Bone marrow transplantation·2014
Same author

Introductory lecture the epidemiology and determinants of obesity in developed and developing countries.

International journal for vitamin and nutrition research. Internationale Zeitschrift fur Vitamin- und Ernahrungsforschung. Journal international de vitaminologie et de nutrition·2007
Same author

Learning nonregular languages: a comparison of simple recurrent networks and LSTM.

Neural computation·2002
Same author

Learning to forget: continual prediction with LSTM.

Neural computation·2000

Area of Science:

  • Artificial Intelligence
  • Machine Learning
  • Deep Learning

Background:

  • Recurrent neural networks (RNNs) struggle with learning long-term dependencies due to vanishing or exploding gradients.
  • Backpropagation through time (BPTT) is computationally intensive and suffers from decaying error backflow.
  • Existing recurrent network algorithms have limitations in solving tasks requiring memory over extended time intervals.

Purpose of the Study:

  • Introduce a novel, efficient gradient-based method, Long Short-Term Memory (LSTM), to address the long-time-lag problem in recurrent networks.
  • Enable networks to learn and store information over significantly extended time intervals.
  • Improve the learning speed and success rate of recurrent neural networks on complex sequential tasks.

Main Methods:

Related Experiment Videos

  • Developed Long Short-Term Memory (LSTM), a recurrent network architecture featuring special units with 'constant error carousels'.
  • Implemented multiplicative gate units to control access to the constant error flow, enabling selective information retention.
  • Truncated gradients strategically to maintain constant error flow without harming learning.
  • Utilized artificial data with diverse pattern representations for experimentation.

Main Results:

  • LSTM successfully bridges time lags exceeding 1000 discrete steps by enforcing constant error flow.
  • Achieved significantly faster learning and higher success rates compared to established methods like BPTT, Elman nets, and others.
  • Demonstrated the ability to solve complex artificial long-time-lag tasks previously unsolvable by recurrent networks.
  • Exhibited O(1) computational complexity per time step and weight, indicating high efficiency.

Conclusions:

  • Long Short-Term Memory (LSTM) provides an effective solution for learning long-term dependencies in sequential data.
  • LSTM networks offer superior performance and efficiency over existing recurrent architectures for tasks with extended time lags.
  • The proposed architecture overcomes critical limitations of gradient-based learning in recurrent neural networks.