Jove
Visualize
Contact Us
JoVE
x logofacebook logolinkedin logoyoutube logo
ABOUT JoVE
OverviewLeadershipBlogJoVE Help Center
AUTHORS
Publishing ProcessEditorial BoardScope & PoliciesPeer ReviewFAQSubmit
LIBRARIANS
TestimonialsSubscriptionsAccessResourcesLibrary Advisory BoardFAQ
RESEARCH
JoVE JournalMethods CollectionsJoVE Encyclopedia of ExperimentsArchive
EDUCATION
JoVE CoreJoVE BusinessJoVE Science EducationJoVE Lab ManualFaculty Resource CenterFaculty Site
Terms & Conditions of Use
Privacy Policy
Policies

Related Experiment Videos

On structure-exploiting trust-region regularized nonlinear least squares algorithms for neural-network learning.

Eiji Mizutani1, James W Demmel

  • 1Department of Computer Science, National Tsing Hua University, Hsinchu 300, Taiwan, ROC. eiji@wayne.cs.nthu.edu.tw

Neural Networks : the Official Journal of the International Neural Network Society
|July 10, 2003
PubMed
Summary
This summary is machine-generated.

Related Concept Videos

You might also read

Related Articles

Articles linked to this work by shared authors, journal, and citation graph.

Sort by
Same author

Skin graft with dermis and appendages generated in vivo by cell competition.

Nature communications·2024
Same author

Functional calcium-responsive parathyroid glands generated using single-step blastocyst complementation.

Proceedings of the National Academy of Sciences of the United States of America·2023
Same author

Chemically defined cytokine-free expansion of human haematopoietic stem cells.

Nature·2023
Same author

Removal of sperm tail using trypsin and pre-activation of oocyte facilitates intracytoplasmic sperm injection in mice and rats.

The Journal of reproduction and development·2022
Same author

Xenotransplantation and interspecies organogenesis: current status and issues.

Frontiers in endocrinology·2022
Same author

Improved development of mouse somatic cell nuclear transfer embryos by chlamydocin analogues, class I and IIa histone deacetylase inhibitors†.

Biology of reproduction·2021

This study presents efficient numerical methods for training neural networks (NNs) by exploiting matrix sparsity. These trust-region algorithms reduce memory and computational costs for complex NN models.

Area of Science:

  • Numerical analysis
  • Machine learning
  • Computational mathematics

Background:

  • Solving nonlinear least squares problems is crucial for training neural networks (NNs).
  • Existing methods can be computationally expensive for large-scale NN models.

Purpose of the Study:

  • To introduce efficient numerical linear algebra approaches for structured nonlinear least squares problems in NNs.
  • To develop algorithms that reduce memory and operation costs for NN learning.

Main Methods:

  • Utilizing trust-region regularization techniques.
  • Exploiting sparsity in Jacobian and Hessian matrices (block-angular or block-arrow structures).
  • Applying direct and iterative trust-region algorithms.

Main Results:

Related Experiment Videos

  • Demonstrated efficiency in memory and operation costs for NN learning algorithms.
  • Analysis of algorithmic strengths and weaknesses using a real-world nonlinear regression problem.
  • Simulation results for multilayer perceptrons (MLP) and neuro-fuzzy modular networks.

Conclusions:

  • The proposed numerical methods offer efficient solutions for training complex neural network models.
  • Sparsity exploitation is key to reducing computational burdens in large-scale NN applications.
  • The algorithms are effective for various NN architectures, including MLPs and mixtures of experts.