Jove
Visualize
Contact Us
JoVE
x logofacebook logolinkedin logoyoutube logo
ABOUT JoVE
OverviewLeadershipBlogJoVE Help Center
AUTHORS
Publishing ProcessEditorial BoardScope & PoliciesPeer ReviewFAQSubmit
LIBRARIANS
TestimonialsSubscriptionsAccessResourcesLibrary Advisory BoardFAQ
RESEARCH
JoVE JournalMethods CollectionsJoVE Encyclopedia of ExperimentsArchive
EDUCATION
JoVE CoreJoVE BusinessJoVE Science EducationJoVE Lab ManualFaculty Resource CenterFaculty Site
Terms & Conditions of Use
Privacy Policy
Policies

Related Experiment Videos

Multiscale optimization in neural nets.

E Mjolsness1, C D Garrett, W L Miranker

  • 1Dept. of Comput. Sci., Yale Univ., New Haven, CT.

IEEE Transactions on Neural Networks
|January 1, 1991
PubMed
Summary
This summary is machine-generated.

Related Concept Videos

You might also read

Related Articles

Articles linked to this work by shared authors, journal, and citation graph.

Sort by
Same author

Combined in silico/in vivo analysis of mechanisms providing for root apical meristem self-organization and maintenance.

Annals of botany·2012
Same author

Competition and cooperation in neuronal processing.

IEEE transactions on neural networks·2008
Same author

Machine learning for science: state of the art and future prospects.

Science (New York, N.Y.)·2001
Same author

Delta-Notch lateral inhibitory patterning in the emergence of ciliated cells in Xenopus: experimental observations and a gene network model.

Pacific Symposium on Biocomputing. Pacific Symposium on Biocomputing·2000
Same author

Convergence properties of the softassign quadratic assignment algorithm.

Neural computation·1999
Same author

A robust point-matching algorithm for autoradiograph alignment.

Medical image analysis·1999
Same journal

Universal perceptron and DNA-like learning algorithm for binary neural networks: LSBF and PBF implementations.

IEEE transactions on neural networks·2013
Same journal

Guest editorial: special section on white box nonlinear prediction models.

IEEE transactions on neural networks·2011
Same journal

Data-based fault-tolerant control of high-speed trains with traction/braking notch nonlinearities and actuator failures.

IEEE transactions on neural networks·2011
Same journal

Guest editorial: special section on data-based control, modeling, and optimization.

IEEE transactions on neural networks·2011
Same journal

Neural network-based multiple robot simultaneous localization and mapping.

IEEE transactions on neural networks·2011
Same journal

Data-driven model-free adaptive control for a class of MIMO nonlinear discrete-time systems.

IEEE transactions on neural networks·2011
See all related articles

This study introduces a multiscale optimization method for neural networks, speeding up convergence by alternating between fine and coarse scales. The novel approach offers a constant speedup factor between two and five, independent of network size.

Area of Science:

  • Computational Science
  • Artificial Intelligence
  • Machine Learning

Background:

  • Large-scale optimization problems in neural networks often suffer from slow convergence.
  • Existing methods may not scale efficiently with problem size or network complexity.

Purpose of the Study:

  • To present a novel multiscale optimization method for neural networks.
  • To accelerate convergence for general objective functions in neural network training.
  • To provide a user-friendly method applicable to various network architectures.

Main Methods:

  • Introduced a smaller, approximate problem at a coarser scale.
  • Alternated relaxation steps between fine-scale and coarse-scale problems.
  • Developed transitions and information flow between scales without disrupting optimization.

Related Experiment Videos

Main Results:

  • Observed a nontrivial speedup factor between two and five, independent of problem size.
  • Demonstrated approximately a fivefold improvement in estimated computational cost.
  • Validated the method's applicability to general objective functions and various neural networks.

Conclusions:

  • The multiscale optimization method significantly accelerates neural network training.
  • The approach is easily applicable, requiring only a partition of fine-scale variables.
  • Further computational cost improvements are anticipated, particularly with problem-specific adaptations.