Jove
Visualize
Contact Us
JoVE
x logofacebook logolinkedin logoyoutube logo
ABOUT JoVE
OverviewLeadershipBlogJoVE Help Center
AUTHORS
Publishing ProcessEditorial BoardScope & PoliciesPeer ReviewFAQSubmit
LIBRARIANS
TestimonialsSubscriptionsAccessResourcesLibrary Advisory BoardFAQ
RESEARCH
JoVE JournalMethods CollectionsJoVE Encyclopedia of ExperimentsArchive
EDUCATION
JoVE CoreJoVE BusinessJoVE Science EducationJoVE Lab ManualFaculty Resource CenterFaculty Site
Terms & Conditions of Use
Privacy Policy
Policies

Related Experiment Videos

Almost linear VC-dimension bounds for piecewise polynomial networks

Bartlett1, Maiorov, Meir

  • 1Australian National University, Systems Engineering Dept, Canberra, AU, 0200. Peter.Bartlett@anu.edu.au.

Neural Computation
|November 6, 1998
PubMed
Summary

We determined bounds for the VC dimension and pseudo-dimension of neural networks with piecewise polynomial activation functions. Fixed-layer networks show WlogW growth, unlike unbounded networks which grow as W-squared.

Related Concept Videos

You might also read

Related Articles

Articles linked to this work by shared authors, journal, and citation graph.

Sort by
Same author

Superlattice formation in mixtures of hard-sphere colloids

Physical review. E, Statistical physics, plasmas, fluids, and related interdisciplinary topics·2000
Same author

Nonlocal electron kinetics and densities of excited atoms in S and P striations

Physical review. E, Statistical physics, plasmas, fluids, and related interdisciplinary topics·2000
Same author

Anomalous proximity effect in underdoped YBa2Cu3O6+x josephson junctions

Physical review letters·2000
Same author

Suppression of plasma turbulence during optimized shear configurations in JET

Physical review letters·2000
Same author

New support vector algorithms

Neural computation·2000
Same author

Strategies of Antiretroviral Therapy in Adults.

The oncologist·1999

Area of Science:

  • Machine Learning
  • Computational Complexity
  • Neural Network Theory

Background:

  • Understanding the generalization capabilities of neural networks is crucial for their effective application.
  • VC dimension and pseudo-dimension are key complexity measures that bound a model's ability to generalize.
  • Feedforward neural networks with piecewise polynomial activation functions are a significant class of models.

Purpose of the Study:

  • To compute upper and lower bounds on the VC dimension and pseudo-dimension for feedforward neural networks with piecewise polynomial activation functions.
  • To analyze how these complexity measures scale with the number of network parameters (W) and layers.
  • To establish error bounds for regression estimation using these networks.

Main Methods:

Related Experiment Videos

  • Theoretical analysis of VC dimension and pseudo-dimension for feedforward neural networks.
  • Derivation of bounds based on the number of parameters (W) and network architecture (fixed vs. unbounded layers).
  • Integration of approximation error rates with complexity bounds.

    • Main Results:

    • For a fixed number of layers, VC dimension and pseudo-dimension grow as W log W, where W is the number of parameters.
    • In contrast, for networks with an unbounded number of layers, these dimensions grow as W^2.
    • Derived error bounds for regression estimation using piecewise polynomial networks with unbounded weights.

    Conclusions:

    • The number of layers significantly impacts the complexity and generalization bounds of neural networks.
    • The derived bounds provide insights into the trade-offs between network depth, width, and expressive power.
    • This work contributes to a deeper theoretical understanding of neural network generalization, particularly for piecewise polynomial models.