Jove
Visualize
Contact Us
JoVE
x logofacebook logolinkedin logoyoutube logo
ABOUT JoVE
OverviewLeadershipBlogJoVE Help Center
AUTHORS
Publishing ProcessEditorial BoardScope & PoliciesPeer ReviewFAQSubmit
LIBRARIANS
TestimonialsSubscriptionsAccessResourcesLibrary Advisory BoardFAQ
RESEARCH
JoVE JournalMethods CollectionsJoVE Encyclopedia of ExperimentsArchive
EDUCATION
JoVE CoreJoVE BusinessJoVE Science EducationJoVE Lab ManualFaculty Resource CenterFaculty Site
Terms & Conditions of Use
Privacy Policy
Policies

Related Experiment Videos

How dependencies between successive examples affect on-line learning

W Wiegerinck1, T Heskes

  • 1Department of Medical Physics and Biophysics, University of Nijmegen, The Netherlands.

Neural Computation
|November 15, 1996
PubMed
Summary
This summary is machine-generated.

Related Concept Videos

You might also read

Related Articles

Articles linked to this work by shared authors, journal, and citation graph.

Sort by
Same author

Role of conduct problems in the relation between Attention-Deficit Hyperactivity disorder, substance use, and gaming.

European neuropsychopharmacology : the journal of the European College of Neuropsychopharmacology·2018
Same author

Attractor learning in synchronized chaotic systems in the presence of unresolved scales.

Chaos (Woodbury, N.Y.)·2018
Same author

Self-organizing maps, vector quantization, and mixture modeling.

IEEE transactions on neural networks·2008
Same author

On 'natural' learning and pruning in multi-layered perceptrons.

Neural computation·2000
Same author

Prediction of bladder outlet obstruction in men with lower urinary tract symptoms using artificial neural networks.

The Journal of urology·1999
Same author

Partial retraining: a new approach to input relevance determination.

International journal of neural systems·1999

Predictable data can hinder neural network learning accuracy. However, example dependencies can help networks escape plateaus, improving the learning process for multilayer perceptrons.

Area of Science:

  • Machine Learning
  • Artificial Intelligence
  • Computational Neuroscience

Background:

  • Online learning dynamics in neural networks are complex.
  • Backpropagation in multilayer perceptrons often encounters plateaus.
  • Dependencies between successive training examples can influence learning.

Purpose of the Study:

  • To analyze the impact of dependent examples on neural network online learning.
  • To define and calculate representation and prediction errors near local minima.
  • To investigate how example dependencies affect learning dynamics, particularly concerning plateaus.

Main Methods:

  • Mathematical analysis of on-line learning dynamics for a broad class of neural networks and learning rules.
  • Definition and calculation of representation error and prediction error.

Related Experiment Videos

  • Study of learning dynamics in the presence of error surface plateaus.
  • Simulations of a multilayer perceptron learning a chaotic time series using backpropagation.

    • Main Results:

    • Higher example predictability leads to increased representation error and less accurate environmental representation.
    • Dependencies between examples can facilitate escaping from plateaus in the error surface.
    • Simulations confirmed that example dependencies aid learning in overcoming plateaus.

    Conclusions:

    • The predictability of training data is a critical factor influencing the accuracy of neural network representations.
    • Leveraging dependencies within sequential data can be a key strategy to mitigate learning stagnation caused by plateaus.
    • Understanding these dynamics is crucial for optimizing neural network training, especially for complex tasks like time series prediction.