Jove
Visualize
Contact Us
JoVE
x logofacebook logolinkedin logoyoutube logo
ABOUT JoVE
OverviewLeadershipBlogJoVE Help Center
AUTHORS
Publishing ProcessEditorial BoardScope & PoliciesPeer ReviewFAQSubmit
LIBRARIANS
TestimonialsSubscriptionsAccessResourcesLibrary Advisory BoardFAQ
RESEARCH
JoVE JournalMethods CollectionsJoVE Encyclopedia of ExperimentsArchive
EDUCATION
JoVE CoreJoVE BusinessJoVE Science EducationJoVE Lab ManualFaculty Resource CenterFaculty Site
Terms & Conditions of Use
Privacy Policy
Policies

Related Experiment Videos

Bayesian neural networks for classification: how useful is the evidence framework?

W D. Penny1, S J. Roberts

  • 1Department of Electrical and Electronic Engineering, Imperial College, London, UK

Neural Networks : the Official Journal of the International Neural Network Society
|March 29, 2003
PubMed
Summary

The Bayesian evidence framework for neural networks is effective for model selection and feature selection, especially with sufficient data. Committees of Bayesian networks offer competitive classification accuracy with minimal human intervention.

Related Concept Videos

You might also read

Related Articles

Articles linked to this work by shared authors, journal, and citation graph.

Sort by
Same author

An empirical evaluation of Bayesian sampling with hybrid Monte Carlo for training neural network classifiers.

Neural networks : the official journal of the International Neural Network Societyยท2003
See all related articles

Area of Science:

  • Machine Learning
  • Artificial Intelligence
  • Computational Statistics

Background:

  • The Bayesian evidence framework offers a principled approach to neural network model selection and hyperparameter inference.
  • Assessing the practical utility of this framework, particularly concerning model selection, automatic relevance determination (ARD), and committee-based approaches, is crucial for its adoption.

Purpose of the Study:

  • To empirically evaluate the Bayesian evidence framework for neural networks across synthetic and real-world classification tasks.
  • To investigate the conditions under which model selection and ARD are effective.
  • To assess the performance of Bayesian network committees.

Main Methods:

  • Utilized four synthetic and four real-world classification datasets.

Related Experiment Videos

  • Focused on three key aspects: model selection, automatic relevance determination (ARD), and committee-based learning.
  • Compared evidence-based model selection with cross-validation.
  • Evaluated ARD for feature selection in networks with numerous hidden units and irrelevant variables.

    • Main Results:

    • Model selection via the evidence criterion requires a substantial data-to-weight ratio (5-10x); cross-validation is a viable alternative when data is abundant.
    • Automatic relevance determination (ARD) proves beneficial for feature selection in networks with many hidden units and datasets with numerous irrelevant variables, also serving as a hard feature selection method.
    • Committees of Bayesian networks achieved classification accuracies comparable to leading methods on real-world data, with minimal human input.

    Conclusions:

    • The Bayesian evidence framework's applicability for model selection is contingent on sufficient training data.
    • ARD is a valuable tool for feature selection in complex neural networks and high-dimensional datasets.
    • Committee-based Bayesian networks provide a robust and efficient approach to classification, requiring minimal expert tuning.