Jove
Visualize
Contact Us
JoVE
x logofacebook logolinkedin logoyoutube logo
ABOUT JoVE
OverviewLeadershipBlogJoVE Help Center
AUTHORS
Publishing ProcessEditorial BoardScope & PoliciesPeer ReviewFAQSubmit
LIBRARIANS
TestimonialsSubscriptionsAccessResourcesLibrary Advisory BoardFAQ
RESEARCH
JoVE JournalMethods CollectionsJoVE Encyclopedia of ExperimentsArchive
EDUCATION
JoVE CoreJoVE BusinessJoVE Science EducationJoVE Lab ManualFaculty Resource CenterFaculty Site
Terms & Conditions of Use
Privacy Policy
Policies

Related Concept Videos

Language Development01:22

Language Development

Children master language quickly and with relative ease, supported by both biological predisposition and reinforcement. B. F. Skinner (1957) proposed that language is learned through reinforcement, while Noam Chomsky (1965) argued that language acquisition mechanisms are biologically determined.
The critical period for language acquisition suggests that the ability to acquire language is at its peak early in life. As people age, this proficiency decreases. Language development begins very...
Language and Cognition01:27

Language and Cognition

Language serves as a bridge between ideas and communication, influencing how individuals perceive and interact with the world. Psychologists have long debated whether language shapes thought or vice versa. This discussion gained grip with Edward Sapir and Benjamin Lee Whorf in the 1940s, who proposed that language determines thought, a concept known as linguistic determinism. They suggested that the vocabulary and structure of a language influence how its speakers think and perceive reality.
LTR Retrotransposons03:08

LTR Retrotransposons

LTR retrotransposons are class I transposable elements with long terminal repeats flanking an internal coding region. These elements are less abundant in mammals compared to other class I transposable elements. About 8 percent of human genomic DNA comprises LTR retrotransposons. Some of the common examples of LTR retrotransposons are Ty elements in yeast and Copia elements in Drosophila.
The internal coding region of LTR retrotransposons and their mechanism of transposition closely resembles a...
Associative Learning01:27

Associative Learning

Associative learning is a fundamental concept in behavioral psychology, wherein a connection is established between two stimuli or events, leading to a learned response. This process is critical in understanding how behaviors are acquired and modified. Conditioning, the mechanism through which associations are formed, can be divided into two main types: classical conditioning and operant conditioning, each elucidating different aspects of associative learning.
Classical conditioning, also known...
Components of Language01:24

Components of Language

Language, whether spoken, signed, or written, consists of specific components: lexicon and grammar. The lexicon is the vocabulary of a language, comprising its words. Grammar is the set of rules used to convey meaning through the lexicon. For example, English grammar adds “-ed” to most verbs to indicate past tense. Words are formed by combining phonemes, which are the basic sound units of a language. Different languages have different sets of phonemes (e.g., “ah” vs. “eh”). Phonemes combine to...
Long-Term Memory01:18

Long-Term Memory

Long-term memory is a relatively permanent type of memory, capable of storing vast amounts of information over extended periods. Its storage capacity is generally considered unlimited.
Long-term memory can be categorized into two primary types: explicit and implicit memory. Explicit memory, also known as declarative memory, involves the conscious recollection of information that we deliberately try to remember, recall, and articulate. This type of memory encompasses specific facts, events, and...

You might also read

Related Articles

Articles linked to this work by shared authors, journal, and citation graph.

Sort by
Same author

Learning to forget: continual prediction with LSTM.

Neural computation·2000
See all related articles

Related Experiment Videos

LSTM recurrent networks learn simple context-free and context-sensitive languages.

F A Gers1, E Schmidhuber

  • 1IDSIA, 6928 Manno, Switzerland. felix@idsia.ch; juergen@idsia.ch

IEEE Transactions on Neural Networks
|February 6, 2008
PubMed
Summary

Long short-term memory (LSTM) networks outperform traditional recurrent neural networks (RNNs) in learning complex languages. LSTMs demonstrate superior performance on context-free and context-sensitive language benchmarks, setting a new standard for RNN capabilities.

Related Experiment Videos

Area of Science:

  • Computational linguistics
  • Machine learning
  • Artificial intelligence

Background:

  • Traditional recurrent neural networks (RNNs) have limitations in learning complex language structures.
  • Long short-term memory (LSTM) networks have shown promise in sequence learning tasks.

Purpose of the Study:

  • To evaluate the performance of LSTM networks on language learning benchmarks.
  • To compare LSTM performance against traditional RNNs and specialized architectures.
  • To investigate LSTM's capability in learning context-sensitive languages.

Main Methods:

  • Utilizing exemplary training sequences for language acquisition.
  • Benchmarking LSTM performance on context-free language tasks.
  • Assessing LSTM variants on the context-sensitive language a(n)b(n)c(n).

Main Results:

  • LSTMs significantly outperform traditional RNNs on context-free language benchmarks.
  • LSTM performance surpasses that of previously established hardwired or specialized architectures.
  • LSTM variants successfully learn a simple context-sensitive language for the first time in RNN research.

Conclusions:

  • LSTMs represent a significant advancement in recurrent neural network capabilities for language learning.
  • The demonstrated ability to learn context-sensitive languages opens new avenues for complex sequence modeling.
  • LSTM networks offer a more robust and versatile approach to grammatical inference.