Jove
Visualize
Contact Us
JoVE
x logofacebook logolinkedin logoyoutube logo
ABOUT JoVE
OverviewLeadershipBlogJoVE Help Center
AUTHORS
Publishing ProcessEditorial BoardScope & PoliciesPeer ReviewFAQSubmit
LIBRARIANS
TestimonialsSubscriptionsAccessResourcesLibrary Advisory BoardFAQ
RESEARCH
JoVE JournalMethods CollectionsJoVE Encyclopedia of ExperimentsArchive
EDUCATION
JoVE CoreJoVE BusinessJoVE Science EducationJoVE Lab ManualFaculty Resource CenterFaculty Site
Terms & Conditions of Use
Privacy Policy
Policies

Related Experiment Videos

Improving generalization performance using double backpropagation.

H Drucker1, Y Le Cun

  • 1ATandT Bell Lab., West Long Branch, NJ.

IEEE Transactions on Neural Networks
|January 1, 1992
PubMed
Summary
This summary is machine-generated.

Related Concept Videos

You might also read

Related Articles

Articles linked to this work by shared authors, journal, and citation graph.

Sort by
Same author

Magnesium alloys as implant materials--principles of property design for Mg-RE alloys.

Acta biomaterialia·2009
Same author

Support vector machines for spam categorization.

IEEE transactions on neural networks·2008
Same author

Characteristics of chloride-dependent incorporation of glutamate into brain membranes argue against a receptor binding site.

Neuropharmacology·1987
Same author

Quisqualate-sensitive, chloride-dependent transport of glutamate into rat brain synaptosomes.

Journal of neuroscience research·1987
Same author

[Epidemiologic study of patients admitted during a 1 year period to psychiatric after care].

Annales medico-psychologiques·1986
Same author

[Difficult exercise of medical confidentiality in psychiatry].

Annales medico-psychologiques·1985
Same journal

Universal perceptron and DNA-like learning algorithm for binary neural networks: LSBF and PBF implementations.

IEEE transactions on neural networks·2013
Same journal

Guest editorial: special section on white box nonlinear prediction models.

IEEE transactions on neural networks·2011
Same journal

Data-based fault-tolerant control of high-speed trains with traction/braking notch nonlinearities and actuator failures.

IEEE transactions on neural networks·2011
Same journal

Guest editorial: special section on data-based control, modeling, and optimization.

IEEE transactions on neural networks·2011
Same journal

Neural network-based multiple robot simultaneous localization and mapping.

IEEE transactions on neural networks·2011
Same journal

Data-driven model-free adaptive control for a class of MIMO nonlinear discrete-time systems.

IEEE transactions on neural networks·2011
See all related articles

Double backpropagation improves generalization by ensuring small input changes do not alter outputs. This method, using an energy function with a Jacobian term, leads to smaller weights and enhanced neural network performance.

Area of Science:

  • Machine Learning
  • Artificial Intelligence
  • Neural Networks

Background:

  • Generalizing from training to test data is crucial for pattern recognition.
  • Standard backpropagation can be sensitive to small input variations, affecting output stability.

Purpose of the Study:

  • To introduce and evaluate a novel training algorithm, double backpropagation, for improved generalization.
  • To enhance the robustness of neural network outputs against minor input perturbations.

Main Methods:

  • Developed double backpropagation by augmenting the standard backpropagation energy function.
  • Incorporated an additional energy term based on the Jacobian of the network's output with respect to its input.
  • Tested the algorithm across diverse neural network architectures and datasets.

Related Experiment Videos

Main Results:

  • Double backpropagation demonstrated significant performance improvements compared to standard backpropagation.
  • The method proved effective across various architectures, including those with prior high performance.
  • Networks trained with double backpropagation exhibited smaller weights, promoting increased neuron output in the linear region.

Conclusions:

  • Double backpropagation offers a robust method for enhancing neural network generalization.
  • The technique's ability to stabilize outputs and modify weight characteristics provides a distinct advantage.
  • This approach is particularly beneficial for complex architectures requiring high generalization capabilities.