Jove
Visualize
Contact Us
JoVE
x logofacebook logolinkedin logoyoutube logo
ABOUT JoVE
OverviewLeadershipBlogJoVE Help Center
AUTHORS
Publishing ProcessEditorial BoardScope & PoliciesPeer ReviewFAQSubmit
LIBRARIANS
TestimonialsSubscriptionsAccessResourcesLibrary Advisory BoardFAQ
RESEARCH
JoVE JournalMethods CollectionsJoVE Encyclopedia of ExperimentsArchive
EDUCATION
JoVE CoreJoVE BusinessJoVE Science EducationJoVE Lab ManualFaculty Resource CenterFaculty Site
Terms & Conditions of Use
Privacy Policy
Policies

Related Experiment Videos

Temporal-kernel recurrent neural networks.

Ilya Sutskever1, Geoffrey Hinton

  • 1Department of Computer Science, University of Toronto, Toronto, Canada. ilya@cs.utoronto.ca

Neural Networks : the Official Journal of the International Neural Network Society
|November 26, 2009
PubMed
Summary
This summary is machine-generated.

Related Concept Videos

Neural Circuits01:25

Neural Circuits

Neural circuits and neuronal pools are two of the main structures found in the nervous system. Neural circuits are networks of neurons that work together to carry out a specific task or process. They consist of interconnected neurons and glial cells, which provide structural and metabolic support.
Neuronal pools are collections of nerve cells with similar functions and interact through chemical and electrical signals. These pools include both interneurons (the central neural circuit nodes that...

You might also read

Related Articles

Articles linked to this work by shared authors, journal, and citation graph.

Sort by
Same author

Managing extreme AI risks amid rapid progress.

Science (New York, N.Y.)·2024
Same author

Robust and data-efficient generalization of self-supervised machine learning for diagnostic imaging.

Nature biomedical engineering·2023
Same author

How to Represent Part-Whole Hierarchies in a Neural Network.

Neural computation·2022
Same author

Backpropagation and the brain.

Nature reviews. Neuroscience·2020
Same author

Deep Learning-A Technology With the Potential to Transform Health Care.

JAMA·2018
Same author

Mastering the game of Go with deep neural networks and tree search.

Nature·2016
Same journal

Exploiting audio-visual modalities in videos: Object detection via multi-stage bilateral coupling network.

Neural networks : the official journal of the International Neural Network Society·2026
Same journal

Reliability-aware modality completion with cross-modal distillation for federated learning with missing modalities.

Neural networks : the official journal of the International Neural Network Society·2026
Same journal

IGFD-Net: Illumination-guided frequency decoupling for polarization image fusion.

Neural networks : the official journal of the International Neural Network Society·2026
Same journal

Multiple-Strategies dung beetle optimizer and its applications in engineering optimization and bankruptcy prediction.

Neural networks : the official journal of the International Neural Network Society·2026
Same journal

Aggregating global-scale pixel-wise forgery cues within a graph.

Neural networks : the official journal of the International Neural Network Society·2026
Same journal

Finite-Time intermittent control for secure synchronization of Neutral-Type stochastic delayed neural networks under aperiodic DoS attacks.

Neural networks : the official journal of the International Neural Network Society·2026
See all related articles

This study introduces the Temporal-Kernel Recurrent Neural Network (TKRNN), a novel model designed to overcome the long-term dependency challenges faced by standard Recurrent Neural Networks (RNNs). The TKRNN effectively solves sequential prediction tasks by developing a stable short-term memory.

Area of Science:

  • Artificial Intelligence
  • Machine Learning
  • Computational Neuroscience

Background:

  • Recurrent Neural Networks (RNNs) are effective for sequential data but struggle with long-term dependencies.
  • Training RNNs on tasks requiring memory over many timesteps remains a significant challenge.
  • Standard RNNs often fail to capture essential information from distant past events.

Purpose of the Study:

  • To address the limitations of standard RNNs in handling long-term dependencies.
  • To introduce a novel RNN variant capable of improved sequential prediction.
  • To evaluate the model's performance on a serial recall task with significant temporal dependencies.

Main Methods:

  • Introduction of the Temporal-Kernel Recurrent Neural Network (TKRNN), a modified RNN architecture.

Related Experiment Videos

  • Focus on a serial recall task requiring the model to remember and reconstruct character sequences.
  • Analysis of the TKRNN's internal state dynamics and memory representation.

    • Main Results:

    • The TKRNN demonstrates a superior ability to manage long-term dependencies compared to standard RNNs.
    • The model successfully solves the serial recall task, indicating effective memory retention.
    • The TKRNN develops a stable short-term memory through the state of its hidden units.

    Conclusions:

    • The Temporal-Kernel Recurrent Neural Network (TKRNN) offers a viable solution for training RNNs on tasks with long-term dependencies.
    • The TKRNN's architecture facilitates the development of robust short-term memory crucial for sequential prediction.
    • This advancement has implications for natural language processing, speech recognition, and other sequential data analysis.