Jove
Visualize
Contact Us
JoVE
x logofacebook logolinkedin logoyoutube logo
ABOUT JoVE
OverviewLeadershipBlogJoVE Help Center
AUTHORS
Publishing ProcessEditorial BoardScope & PoliciesPeer ReviewFAQSubmit
LIBRARIANS
TestimonialsSubscriptionsAccessResourcesLibrary Advisory BoardFAQ
RESEARCH
JoVE JournalMethods CollectionsJoVE Encyclopedia of ExperimentsArchive
EDUCATION
JoVE CoreJoVE BusinessJoVE Science EducationJoVE Lab ManualFaculty Resource CenterFaculty Site
Terms & Conditions of Use
Privacy Policy
Policies

Related Experiment Videos

Energy-Efficient LSTM Networks for Online Learning.

Tolga Ergen, Ali H Mirza, Suleyman Serdar Kozat

    IEEE Transactions on Neural Networks and Learning Systems
    |September 20, 2019
    PubMed
    Summary

    This study introduces an energy-efficient regression model using Long Short-Term Memory (LSTM) networks for variable-length data. The novel approach significantly reduces complexity and enhances performance through efficient training algorithms.

    Related Concept Videos

    Long-term Potentiation01:35

    Long-term Potentiation

    58.4K
    Long-term potentiation, or LTP, is one of the ways by which synaptic plasticity—changes in the strength of chemical synapses—can occur in the brain. LTP is the process of synaptic strengthening that occurs over time between pre- and postsynaptic neuronal connections. The synaptic strengthening of LTP works in opposition to the synaptic weakening of long-term depression (LTD) and together are the main mechanisms that underlie learning and memory.
    58.4K
    Introduction to Learning01:18

    Introduction to Learning

    973
    Learning is the process of acquiring knowledge or skills through practice or experience, leading to long-lasting behavioral changes. This acquisition occurs through interaction with the environment and requires practice or experience. For instance, mastering a skill such as surfing requires considerable practice and experience, highlighting the essential role of repeated interactions with the environment in learning.
    In contrast to learned behaviors, unlearned behaviors such as crying, sexual...
    973

    You might also read

    Related Articles

    Articles linked to this work by shared authors, journal, and citation graph.

    Sort by
    Same author

    Spatiotemporal Sequence Prediction With Point Processes and Self-Organizing Decision Trees.

    IEEE transactions on neural networks and learning systems·2021
    Same author

    Markovian RNN: An Adaptive Time Series Prediction Network With HMM-Based Switching for Nonstationary Environments.

    IEEE transactions on neural networks and learning systems·2021
    Same author

    Multi-Label Sentiment Analysis on 100 Languages With Dynamic Weighting for Label Imbalance.

    IEEE transactions on neural networks and learning systems·2021
    Same author

    Achieving Online Regression Performance of LSTMs With Simple RNNs.

    IEEE transactions on neural networks and learning systems·2021
    Same author

    Online Anomaly Detection With Bandwidth Optimized Hierarchical Kernel Density Estimators.

    IEEE transactions on neural networks and learning systems·2020
    Same author

    Unsupervised Anomaly Detection With LSTM Neural Networks.

    IEEE transactions on neural networks and learning systems·2019

    Area of Science:

    • Machine Learning
    • Deep Learning
    • Computational Efficiency

    Background:

    • Online regression with variable-length data presents computational challenges.
    • Existing Long Short-Term Memory (LSTM) network architectures can be complex and resource-intensive.

    Purpose of the Study:

    • To develop an energy-efficient regression structure for variable-length data using LSTM networks.
    • To introduce effective online training algorithms for the proposed structure.
    • To reduce the computational complexity and parameter count of LSTM-based regression models.

    Main Methods:

    • A generic LSTM-based regression structure for variable-length input sequences was developed.
    • Regular multiplication operations were replaced with an energy-efficient ef-operator.

    Related Experiment Videos

  • Weight matrices in the LSTM network were factorized to reduce the number of trainable parameters.
  • Online training algorithms based on Stochastic Gradient Descent (SGD) and Exponentiated Gradient (EG) were introduced.
  • An energy-efficient Gated Recurrent Unit (GRU) network was also simulated.

    • Main Results:

    • Significant performance gains were observed compared to conventional methods.
    • Substantial complexity reductions were achieved through the proposed energy-efficient operator and matrix factorization.
    • The developed online training algorithms demonstrated high efficiency and effectiveness.

    Conclusions:

    • The proposed LSTM-based regression structure offers a computationally efficient and effective solution for variable-length data in an online setting.
    • The integration of the ef-operator and weight factorization leads to reduced complexity and energy consumption.
    • The developed online training algorithms are highly suitable for the efficient learning of these optimized network structures.