Jove
Visualize
Contact Us
JoVE
x logofacebook logolinkedin logoyoutube logo
ABOUT JoVE
OverviewLeadershipBlogJoVE Help Center
AUTHORS
Publishing ProcessEditorial BoardScope & PoliciesPeer ReviewFAQSubmit
LIBRARIANS
TestimonialsSubscriptionsAccessResourcesLibrary Advisory BoardFAQ
RESEARCH
JoVE JournalMethods CollectionsJoVE Encyclopedia of ExperimentsArchive
EDUCATION
JoVE CoreJoVE BusinessJoVE Science EducationJoVE Lab ManualFaculty Resource CenterFaculty Site
Terms & Conditions of Use
Privacy Policy
Policies

Related Experiment Videos

Invariance priors for Bayesian feed-forward neural networks.

Udo V Toussaint1, Silvio Gori, Volker Dose

  • 1Centre for Interdisciplinary Plasma Science, Max-Planck-Institut für Plasmaphysik, EURATOM Association, Boltzmannstr. 2, D-85748 Garching, Germany.

Neural Networks : the Official Journal of the International Neural Network Society
|April 4, 2006
PubMed
Summary

This study introduces a novel Bayesian approach to regularize neural networks (NN), addressing overfitting and improving generalization. The method provides a general prior for feed-forward networks and enables cell pruning.

Related Concept Videos

You might also read

Related Articles

Articles linked to this work by shared authors, journal, and citation graph.

Sort by
Same author

Nut crop yield records show that budbreak-based chilling requirements may not reflect yield decline chill thresholds.

International journal of biometeorology·2014
Same author

Detecting nonlinear response of spring phenology to climate change by Bayesian analysis.

Global change biology·2013
Same author

Bayesian analysis of the species-specific lengthening of the growing season in two European countries and the influence of an insect pest.

International journal of biometeorology·2007
Same author

Bayesian neural-networks-based evaluation of binary speckle data.

Applied optics·2004

Area of Science:

  • Artificial Intelligence
  • Machine Learning
  • Computational Neuroscience

Background:

  • Neural networks (NN) offer flexibility but suffer from overfitting and poor generalization.
  • Existing regularization methods for NN often lack theoretical grounding, relying on ad hoc arguments.

Purpose of the Study:

  • To derive a general prior for feed-forward neural networks using Bayesian probability theory.
  • To enhance the regularization of neural networks and improve their generalization capabilities.
  • To introduce a principled method for cell pruning in neural networks.

Main Methods:

  • The study employs the principle of transformation invariance.
  • A general prior is derived based on Bayesian probability theory for feed-forward networks.

Related Experiment Videos

  • Bayesian model comparison is used to determine optimal network configurations.

    • Main Results:

    • A novel, theoretically grounded prior for neural network regularization is presented.
    • The derived prior effectively addresses overfitting and enhances network generalization.
    • The approach facilitates effective cell pruning, optimizing network structure.

    Conclusions:

    • The Bayesian approach offers a principled framework for regularizing neural networks.
    • This method improves network performance by mitigating overfitting and enhancing generalization.
    • The derived prior provides a foundation for advanced techniques like cell pruning.