Jove
Visualize
Contact Us
JoVE
x logofacebook logolinkedin logoyoutube logo
ABOUT JoVE
OverviewLeadershipBlogJoVE Help Center
AUTHORS
Publishing ProcessEditorial BoardScope & PoliciesPeer ReviewFAQSubmit
LIBRARIANS
TestimonialsSubscriptionsAccessResourcesLibrary Advisory BoardFAQ
RESEARCH
JoVE JournalMethods CollectionsJoVE Encyclopedia of ExperimentsArchive
EDUCATION
JoVE CoreJoVE BusinessJoVE Science EducationJoVE Lab ManualFaculty Resource CenterFaculty Site
Terms & Conditions of Use
Privacy Policy
Policies

Related Experiment Videos

Learning to forget: continual prediction with LSTM.

F A Gers1, J Schmidhuber, F Cummins

  • 1IDSIA, Lugano, Switzerland.

Neural Computation
|October 14, 2000
PubMed
Summary
This summary is machine-generated.

Related Concept Videos

You might also read

Related Articles

Articles linked to this work by shared authors, journal, and citation graph.

Sort by
Same author

Paediatric hospital admissions for psychiatric and psychosocial reasons during the first year of the COVID-19 pandemic.

International review of psychiatry (Abingdon, England)·2022
Same author

African swine fever: a global factor affecting agricultural markets over the medium term.

Revue scientifique et technique (International Office of Epizootics)·2022
Same author

Optimising traceability in trade for live animals and animal products with digital technologies.

Revue scientifique et technique (International Office of Epizootics)·2020
Same author

Salvage therapy with everolimus reduces the severity of treatment-refractory chronic GVHD without impairing disease control: a dual center retrospective analysis.

Bone marrow transplantation·2014
Same author

Surgery of the femur in HIV positive patients: a retrospective review from 2005 to 2011.

Irish journal of medical science·2014
Same author

Use of lean principals to improve flow of patients with fractured neck of femur--the HOPE study.

Irish medical journal·2014

Long short-term memory (LSTM) networks struggle with continuous data streams. A new adaptive forget gate allows LSTM cells to self-reset, solving problems that previously caused breakdowns.

Area of Science:

  • Artificial Intelligence
  • Machine Learning
  • Deep Learning

Background:

  • Recurrent Neural Networks (RNNs) have limitations in processing sequential data.
  • Long Short-Term Memory (LSTM) networks offer improvements over traditional RNNs but face challenges with continuous input streams.
  • Unsegmented continuous data can lead to indefinite state growth and network failure in standard LSTMs.

Purpose of the Study:

  • To identify and address the limitations of standard LSTM networks in processing continuous, unsegmented input data.
  • To introduce a novel mechanism for LSTMs to manage their internal state during continuous data processing.
  • To enhance the robustness and applicability of LSTM networks to real-world sequential data challenges.

Main Methods:

  • Development of an adaptive 'forget gate' mechanism for LSTM cells.

Related Experiment Videos

  • Implementation of the forget gate to enable self-resetting capabilities within the LSTM.
  • Evaluation of the modified LSTM on benchmark problems, including continual versions where standard LSTMs fail.

    • Main Results:

    • Standard LSTM networks fail to solve continual versions of benchmark problems due to unmanaged state growth.
    • LSTM networks equipped with the adaptive forget gate successfully solve these continual problems.
    • The forget gate allows LSTM cells to learn optimal times for resetting their internal state, preventing breakdown.

    Conclusions:

    • The adaptive forget gate is an effective solution for enabling LSTMs to process continuous input streams.
    • This enhancement significantly improves LSTM performance and robustness in scenarios without explicit data segmentation.
    • The modified LSTM offers a more elegant and reliable approach for handling long-term dependencies in sequential data.