Jove
Visualize
Contact Us
JoVE
x logofacebook logolinkedin logoyoutube logo
ABOUT JoVE
OverviewLeadershipBlogJoVE Help Center
AUTHORS
Publishing ProcessEditorial BoardScope & PoliciesPeer ReviewFAQSubmit
LIBRARIANS
TestimonialsSubscriptionsAccessResourcesLibrary Advisory BoardFAQ
RESEARCH
JoVE JournalMethods CollectionsJoVE Encyclopedia of ExperimentsArchive
EDUCATION
JoVE CoreJoVE BusinessJoVE Science EducationJoVE Lab ManualFaculty Resource CenterFaculty Site
Terms & Conditions of Use
Privacy Policy
Policies

Related Experiment Videos

Pruning recurrent neural networks for improved generalization performance.

C L Giles1, C W Omlin

  • 1NEC Res. Inst., Princeton, NJ.

IEEE Transactions on Neural Networks
|January 1, 1994
PubMed
Summary
This summary is machine-generated.

Related Concept Videos

You might also read

Related Articles

Articles linked to this work by shared authors, journal, and citation graph.

Sort by
Same author

Poster presentations at medical conferences: an effective way of disseminating research?

Clinical medicine (London, England)·2011
Same author

Optical computing: introduction by the feature editors.

Applied optics·2010
Same author

Learning, invariance, and generalization in high-order neural networks.

Applied optics·2010
Same author

Multiplexed coherent optical processor for calculating generalized moments.

Optics letters·2009
Same author

Neural networks and hybrid intelligent models: foundations, theory, and applications.

IEEE transactions on neural networks·2008
Same author

A machine learning method for extracting symbolic knowledge from recurrent neural networks.

Neural computation·2004
Same journal

Universal perceptron and DNA-like learning algorithm for binary neural networks: LSBF and PBF implementations.

IEEE transactions on neural networks·2013
Same journal

Guest editorial: special section on white box nonlinear prediction models.

IEEE transactions on neural networks·2011
Same journal

Data-based fault-tolerant control of high-speed trains with traction/braking notch nonlinearities and actuator failures.

IEEE transactions on neural networks·2011
Same journal

Guest editorial: special section on data-based control, modeling, and optimization.

IEEE transactions on neural networks·2011
Same journal

Neural network-based multiple robot simultaneous localization and mapping.

IEEE transactions on neural networks·2011
Same journal

Data-driven model-free adaptive control for a class of MIMO nonlinear discrete-time systems.

IEEE transactions on neural networks·2011
See all related articles

This study introduces a pruning heuristic to optimize recurrent neural network architecture. This method enhances generalization performance and extracts more consistent rules from trained networks.

Area of Science:

  • Artificial Intelligence
  • Machine Learning
  • Computational Neuroscience

Background:

  • Determining optimal neural network architecture, particularly for recurrent neural networks (RNNs), remains a challenge.
  • Existing methods lack general approaches for estimating key architectural parameters like hidden layers, neuron counts, or weight sizes.
  • Poor architectural choices can hinder generalization performance in trained models.

Purpose of the Study:

  • To present a novel, simple pruning heuristic for recurrent neural networks.
  • To improve the generalization performance of trained recurrent neural networks.
  • To demonstrate that extracted rules from pruned networks align better with target grammar rules.

Main Methods:

  • A pruning heuristic was developed and applied to fully recurrent neural networks.

Related Experiment Videos

  • Networks were trained on positive and negative strings of regular grammars.
  • The heuristic involved pruning and retraining the networks to refine their architecture.
  • Simulations were conducted on a 10-state random grammar and an 8-state triple-parity grammar.

    • Main Results:

    • The pruning heuristic significantly improved the generalization performance of trained recurrent neural networks.
    • Rules extracted from networks trained with the heuristic were more consistent with the learned grammar rules.
    • The pruning method demonstrated superior generalization performance compared to traditional weight decay techniques.
    • Effectiveness was validated on two distinct regular grammars.

    Conclusions:

    • The proposed pruning heuristic offers an effective strategy for optimizing recurrent neural network architecture.
    • This method enhances model generalization and the interpretability of learned rules.
    • The heuristic provides a valuable alternative to existing regularization techniques like weight decay.