Jove
Visualize
Contact Us
JoVE
x logofacebook logolinkedin logoyoutube logo
ABOUT JoVE
OverviewLeadershipBlogJoVE Help Center
AUTHORS
Publishing ProcessEditorial BoardScope & PoliciesPeer ReviewFAQSubmit
LIBRARIANS
TestimonialsSubscriptionsAccessResourcesLibrary Advisory BoardFAQ
RESEARCH
JoVE JournalMethods CollectionsJoVE Encyclopedia of ExperimentsArchive
EDUCATION
JoVE CoreJoVE BusinessJoVE Science EducationJoVE Lab ManualFaculty Resource CenterFaculty Site
Terms & Conditions of Use
Privacy Policy
Policies

Related Experiment Videos

Adding learning to cellular genetic algorithms for training recurrent neural networks.

K W Ku1, M W Mak, W C Siu

  • 1Department of Electronic and Information Engineering, The Hong Kong Polytechnic University, Hong Kong.

IEEE Transactions on Neural Networks
|February 7, 2008
PubMed
Summary
This summary is machine-generated.

Related Concept Videos

You might also read

Related Articles

Articles linked to this work by shared authors, journal, and citation graph.

Sort by
Same author

A deblocking technique for block-transform compressed image using wavelet transform modulus maxima.

IEEE transactions on image processing : a publication of the IEEE Signal Processing Society·2008
Same author

A class of competitive learning models which avoids neuron underutilization problem.

IEEE transactions on neural networks·2008
Same author

An efficient search strategy for block motion estimation using image features.

IEEE transactions on image processing : a publication of the IEEE Signal Processing Society·2008
Same author

Estimation of elliptical basis function parameters by the EM algorithm with application to speaker verification.

IEEE transactions on neural networks·2008
Same author

BiVisu: software tool for bicluster detection and visualization.

Bioinformatics (Oxford, England)·2007
Same author

A comparative study on kernel-based probabilistic neural networks for speaker verification.

International journal of neural systems·2002
Same journal

Universal perceptron and DNA-like learning algorithm for binary neural networks: LSBF and PBF implementations.

IEEE transactions on neural networks·2013
Same journal

Data-based fault-tolerant control of high-speed trains with traction/braking notch nonlinearities and actuator failures.

IEEE transactions on neural networks·2011
Same journal

Neural network-based multiple robot simultaneous localization and mapping.

IEEE transactions on neural networks·2011
Same journal

Data-driven model-free adaptive control for a class of MIMO nonlinear discrete-time systems.

IEEE transactions on neural networks·2011
Same journal

Data-based identification and control of nonlinear systems via piecewise affine approximation.

IEEE transactions on neural networks·2011
Same journal

Data-core-based fuzzy min-max neural network for pattern classification.

IEEE transactions on neural networks·2011
See all related articles

This study introduces a hybrid optimization algorithm combining local search and cellular genetic algorithms for training recurrent neural networks. Embedding the delta rule within cellular genetic algorithms proved to be the fastest training method.

Area of Science:

  • Artificial Intelligence
  • Machine Learning
  • Computational Neuroscience

Background:

  • Recurrent Neural Networks (RNNs) are powerful for sequential data but training them, especially for long-term dependencies, is challenging.
  • Hybrid optimization approaches can potentially improve training efficiency by combining global search (genetic algorithms) with local refinement (learning).

Purpose of the Study:

  • To propose and evaluate a hybrid optimization algorithm for training RNNs, integrating cellular genetic algorithms with various local learning methods.
  • To compare the effectiveness of Lamarckian and Baldwinian learning mechanisms within this hybrid framework.
  • To identify the most efficient learning strategy for optimizing RNN training.

Main Methods:

  • A hybrid algorithm combining cellular genetic algorithms (GAs) with local search (individual learning) was developed.

Related Experiment Videos

  • RNN weights were encoded as chromosomes, with reproduction occurring on a grid.
  • Learning methods incorporated included real-time recurrent learning (RTRL) and the delta rule, evaluated under Lamarckian and Baldwinian paradigms.

    • Main Results:

    • Baldwinian learning demonstrated inefficiency in assisting cellular GAs, suggesting a mismatch between genotypic and phenotypic changes.
    • Lamarckian mechanisms generally improved the reduction in generations for optimal networks, though not always actual training time.
    • Embedding the delta rule within the cellular GA framework emerged as the fastest training method.

    Conclusions:

    • The choice of learning mechanism significantly impacts the efficiency of hybrid GA-based RNN training.
    • Lamarckian learning is more effective than Baldwinian learning for this hybrid approach.
    • The delta rule, when integrated with cellular GAs, offers the most time-efficient training, with optimal learning intensity being crucial for benefit.