Jove
Visualize
Contact Us
JoVE
x logofacebook logolinkedin logoyoutube logo
ABOUT JoVE
OverviewLeadershipBlogJoVE Help Center
AUTHORS
Publishing ProcessEditorial BoardScope & PoliciesPeer ReviewFAQSubmit
LIBRARIANS
TestimonialsSubscriptionsAccessResourcesLibrary Advisory BoardFAQ
RESEARCH
JoVE JournalMethods CollectionsJoVE Encyclopedia of ExperimentsArchive
EDUCATION
JoVE CoreJoVE BusinessJoVE Science EducationJoVE Lab ManualFaculty Resource CenterFaculty Site
Terms & Conditions of Use
Privacy Policy
Policies

Related Experiment Videos

Adaptive natural gradient learning algorithms for various stochastic models.

H Park1, S I Amari, K Fukumizu

  • 1Department of Computer Science, Yonsei University, Seoul, South Korea. hypark@csai.yonsei.ac.kr

Neural Networks : the Official Journal of the International Neural Network Society
|January 11, 2001
PubMed
Summary

The adaptive natural gradient method improves learning speed by estimating the Fisher information matrix inverse. This approach is extended to various regression and classification models, showing practical advantages in experiments.

Related Concept Videos

You might also read

Related Articles

Articles linked to this work by shared authors, journal, and citation graph.

Sort by
Same author

Search for CP violation in τ±→K(S)0π±ντ decays at Belle.

Physical review letters·2011
Same author

Search for three-jet resonances in pp collisions at square root(s)=7  TeV.

Physical review letters·2011
Same author

Relationship between body-mass index and serum folate concentrations in pregnant women.

European journal of clinical nutrition·2011
Same author

Measurement of the t-channel single top quark production cross section in pp collisions at √s=7  TeV.

Physical review letters·2011
Same author

Observation of transverse polarization asymmetries of charged pion pairs in e+e- annihilation near √s = 10.58 GeV.

Physical review letters·2011
Same author

Indications of suppression of excited Υ states in Pb-Pb collisions at √(s(NN))=2.76 TeV.

Physical review letters·2011

Area of Science:

  • Machine Learning
  • Optimization Algorithms

Background:

  • Standard gradient descent suffers from slow learning speeds due to plateaus.
  • Natural gradient method offers ideal dynamic behavior but requires computationally intensive Fisher information matrix inverse calculation.
  • Existing adaptive methods estimate the Fisher information matrix inverse for natural gradient learning.

Purpose of the Study:

  • To extend the adaptive natural gradient method to a broader range of stochastic models.
  • To provide explicit formulations for adaptive natural gradients in regression and classification.
  • To demonstrate the practical benefits of the proposed algorithms.

Main Methods:

  • Developing an adaptive method to estimate the inverse Fisher information matrix.
  • Extending the adaptive natural gradient to regression with arbitrary noise models.

Related Experiment Videos

  • Applying the adaptive natural gradient to classification with an arbitrary number of classes.

    • Main Results:

    • The adaptive natural gradient method is successfully extended to diverse stochastic models.
    • Explicit forms of the adaptive natural gradient were derived for the targeted models.
    • Computational experiments on benchmark problems confirmed the practical advantages of the algorithms.

    Conclusions:

    • The adaptive natural gradient learning method offers a viable solution for overcoming the limitations of standard gradient descent.
    • The extended method provides efficient and practical solutions for complex machine learning tasks.
    • The proposed algorithms demonstrate significant improvements in learning dynamics and performance.