Jove
Visualize
Contact Us
JoVE
x logofacebook logolinkedin logoyoutube logo
ABOUT JoVE
OverviewLeadershipBlogJoVE Help Center
AUTHORS
Publishing ProcessEditorial BoardScope & PoliciesPeer ReviewFAQSubmit
LIBRARIANS
TestimonialsSubscriptionsAccessResourcesLibrary Advisory BoardFAQ
RESEARCH
JoVE JournalMethods CollectionsJoVE Encyclopedia of ExperimentsArchive
EDUCATION
JoVE CoreJoVE BusinessJoVE Science EducationJoVE Lab ManualFaculty Resource CenterFaculty Site
Terms & Conditions of Use
Privacy Policy
Policies

Related Experiment Videos

Training recurrent networks by Evolino.

Jürgen Schmidhuber1, Daan Wierstra, Matteo Gagliolo

  • 1IDSIA, 6928 Manno (Lugano), Switzerland. juergen@idsia.ch

Neural Computation
|February 15, 2007
PubMed
Summary
This summary is machine-generated.

Related Concept Videos

You might also read

Related Articles

Articles linked to this work by shared authors, journal, and citation graph.

Sort by
Same author

Atmospheric pressure influence on the charge collection efficiency of air-vented ionization chambers in ultra-high dose per pulse electron beams for FLASH radiotherapy.

Physics in medicine and biology·2026
Same author

First experimental characterization of a low-pressure nitrogen filled parallel-plate ionization chamber for UHDP electron beam dosimetry.

Medical physics·2026
Same author

Numerical simulations of charge transport in low-pressure noble gases for ultra-high dose per pulse applications.

Physics in medicine and biology·2026
Same author

A portable graphite calorimeter for onsite reference dosimetry and beam quality correction factor determination.

Journal of applied clinical medical physics·2025
Same author

Evaluation of the two-voltage method for parallel-plate ionization chambers irradiated with pulsed beams.

Medical physics·2025
Same author

Accelerating the inference of string generation-based chemical reaction models for industrial applications.

Journal of cheminformatics·2025

Evolino, a new method for recurrent neural networks (RNNs), evolves hidden node weights and optimizes linear outputs. This approach overcomes local minima challenges, outperforming traditional methods and Echo State networks in complex tasks.

Area of Science:

  • Artificial Intelligence
  • Machine Learning
  • Computational Neuroscience

Background:

  • Gradient-based Long Short-Term Memory (LSTM) recurrent neural networks (RNNs) have advanced task learnability.
  • Training RNNs can be hindered by local minima, limiting gradient information utility.

Purpose of the Study:

  • Introduce EVOlution of systems with LINear Outputs (Evolino), a novel method for training RNNs.
  • Address limitations of gradient-based methods in scenarios with numerous local minima.

Main Methods:

  • Evolino evolves weights to nonlinear hidden nodes of RNNs.
  • Optimal linear mappings from hidden state to output are computed using methods like pseudo-inverse-based linear regression.
  • Evolutionary recurrent support vector machines are derived using quadratic programming to maximize margin.

Related Experiment Videos

Main Results:

  • Evolino-based LSTM successfully tackles tasks intractable for Echo State networks.
  • Achieves superior accuracy in continuous function generation compared to conventional gradient descent RNNs, including gradient-based LSTM.

Conclusions:

  • Evolino offers a robust alternative to gradient-based training for RNNs, especially when local minima are prevalent.
  • Demonstrates enhanced performance and broader applicability in complex sequence modeling and function generation tasks.