Jove
Visualize
Contact Us
JoVE
x logofacebook logolinkedin logoyoutube logo
ABOUT JoVE
OverviewLeadershipBlogJoVE Help Center
AUTHORS
Publishing ProcessEditorial BoardScope & PoliciesPeer ReviewFAQSubmit
LIBRARIANS
TestimonialsSubscriptionsAccessResourcesLibrary Advisory BoardFAQ
RESEARCH
JoVE JournalMethods CollectionsJoVE Encyclopedia of ExperimentsArchive
EDUCATION
JoVE CoreJoVE BusinessJoVE Science EducationJoVE Lab ManualFaculty Resource CenterFaculty Site
Terms & Conditions of Use
Privacy Policy
Policies

Related Experiment Videos

Optimal pruning in neural networks.

D M Barbato1, O Kinouchi

  • 1Universidade Paulista, Avenida Comendador Enzo Ferrari 280, CEP 13043-055, Campinas, SP, Brazil.

Physical Review. E, Statistical Physics, Plasmas, Fluids, and Related Interdisciplinary Topics
|January 4, 2001
PubMed
Summary

We analyzed pruning in simple perceptrons using statistical mechanics. Optimal pruning of small synaptic weights enhances network performance only after a critical learning period, revealing insights into biological pruning.

Related Concept Videos

You might also read

Related Articles

Articles linked to this work by shared authors, journal, and citation graph.

Sort by
Same author

Critical avalanches and subsampling in map-based neural networks coupled with noisy synapses.

Physical review. E, Statistical, nonlinear, and soft matter physics·2013
Same author

A brief history of excitable map-based neurons and neural networks.

Journal of neuroscience methods·2013
Same author

Scaling law for the transient behavior of type-II neuron models.

Physical review. E, Statistical, nonlinear, and soft matter physics·2007
Same author

Robustness of scale invariance in models with self-organized criticality.

Physical review. E, Statistical physics, plasmas, fluids, and related interdisciplinary topics·2002
Same author

Deterministic walks in random media.

Physical review letters·2001
Same author

An educational program for training physicians in personnel recruitment.

Family medicine·1988

Area of Science:

  • Artificial Intelligence
  • Machine Learning Theory
  • Computational Neuroscience

Background:

  • Supervised learning in artificial neural networks involves training synaptic weights.
  • Pruning strategies aim to optimize network performance by removing redundant weights.
  • Understanding weight distributions is crucial for effective pruning.

Purpose of the Study:

  • To analytically determine optimal pruning strategies for simple perceptrons.
  • To investigate the relationship between learning performance and pruning effectiveness.
  • To explore potential implications for biological synaptic pruning.

Main Methods:

  • Utilizing the statistical mechanics approach to learning theory.
  • Calculating the post-training distribution of synaptic weights (P(J)).
  • Deriving an optimal pruning threshold as a function of learning overlap and relevant weight fraction.

Main Results:

  • The distribution of synaptic weights is independent of the specific learning algorithm.
  • An optimal pruning threshold, theta(opt)(rho(0), kappa), was derived.
  • Effective pruning, enhancing network performance, occurs only above a critical learning overlap (rho(c)(kappa)).

Conclusions:

  • The elimination of weak synapses is beneficial only after a sufficient learning period.
  • The derived pruning strategy offers a theoretical framework for optimizing perceptron performance.
  • Findings may provide insights into the mechanisms of synaptic pruning in biological neural networks.

Related Experiment Videos