Search research articles

Search research articles

Related Experiment Videos

How dependencies between successive examples affect on-line learning

W Wiegerinck¹, T Heskes

¹Department of Medical Physics and Biophysics, University of Nijmegen, The Netherlands.

Neural Computation

|November 15, 1996

Summary

This summary is machine-generated.

Related Concept Videos

You might also read

Related Articles

Articles linked to this work by shared authors, journal, and citation graph.

Sort by

Same author

Role of conduct problems in the relation between Attention-Deficit Hyperactivity disorder, substance use, and gaming.

European neuropsychopharmacology : the journal of the European College of Neuropsychopharmacology·2018

Same author

Attractor learning in synchronized chaotic systems in the presence of unresolved scales.

Chaos (Woodbury, N.Y.)·2018

Same author

Self-organizing maps, vector quantization, and mixture modeling.

IEEE transactions on neural networks·2008

Same author

On 'natural' learning and pruning in multi-layered perceptrons.

Neural computation·2000

Same author

Prediction of bladder outlet obstruction in men with lower urinary tract symptoms using artificial neural networks.

The Journal of urology·1999

Same author

Partial retraining: a new approach to input relevance determination.

International journal of neural systems·1999

ABOUT JoVE

Overview Leadership Blog JoVE Help Center

AUTHORS

Publishing Process Editorial Board Scope & Policies Peer Review FAQ Submit

LIBRARIANS

Testimonials Subscriptions Access Resources Library Advisory Board FAQ

RESEARCH

JoVE Journal Methods Collections JoVE Encyclopedia of Experiments Archive

EDUCATION

JoVE Core JoVE Business JoVE Science Education JoVE Lab Manual Faculty Resource Center Faculty Site

Terms & Conditions of Use

Predictable data can hinder neural network learning accuracy. However, example dependencies can help networks escape plateaus, improving the learning process for multilayer perceptrons.

Area of Science:

Machine Learning
Artificial Intelligence
Computational Neuroscience

Background:

Online learning dynamics in neural networks are complex.
Backpropagation in multilayer perceptrons often encounters plateaus.
Dependencies between successive training examples can influence learning.

Purpose of the Study:

To analyze the impact of dependent examples on neural network online learning.
To define and calculate representation and prediction errors near local minima.
To investigate how example dependencies affect learning dynamics, particularly concerning plateaus.

Main Methods:

Mathematical analysis of on-line learning dynamics for a broad class of neural networks and learning rules.
Definition and calculation of representation error and prediction error.

Related Experiment Videos

Study of learning dynamics in the presence of error surface plateaus.

Simulations of a multilayer perceptron learning a chaotic time series using backpropagation.

Main Results:

Higher example predictability leads to increased representation error and less accurate environmental representation.
Dependencies between examples can facilitate escaping from plateaus in the error surface.
Simulations confirmed that example dependencies aid learning in overcoming plateaus.

Conclusions:

The predictability of training data is a critical factor influencing the accuracy of neural network representations.
Leveraging dependencies within sequential data can be a key strategy to mitigate learning stagnation caused by plateaus.
Understanding these dynamics is crucial for optimizing neural network training, especially for complex tasks like time series prediction.