Search research articles

Search research articles

Related Experiment Videos

Neural network pruning with Tukey-Kramer multiple comparison procedure.

Donald E Duckro¹, Dennis W Quinn, Samuel J Gardner

¹Air Force Institute of Technology, Department of Mathematics and Statistics, Wright-Patterson Air Force Base, Ohio 45433, U.S.A. duckro@yahoo.com

Neural Computation

|April 26, 2002

Summary

This summary is machine-generated.

Related Concept Videos

You might also read

Related Articles

Articles linked to this work by shared authors, journal, and citation graph.

Sort by

Same author

Reactivity of an aromatic sigma,sigma,sigma-triradical: the 2,4,6-tridehydropyridinium cation.

Angewandte Chemie (International ed. in English)·2007

See all related articles

ABOUT JoVE

Overview Leadership Blog JoVE Help Center

AUTHORS

Publishing Process Editorial Board Scope & Policies Peer Review FAQ Submit

LIBRARIANS

Testimonials Subscriptions Access Resources Library Advisory Board FAQ

RESEARCH

JoVE Journal Methods Collections JoVE Encyclopedia of Experiments Archive

EDUCATION

JoVE Core JoVE Business JoVE Science Education JoVE Lab Manual Faculty Resource Center Faculty Site

Terms & Conditions of Use

This study introduces a new statistical method for pruning neural networks, improving their generalization ability. The bootstrap algorithm and statistical comparisons effectively remove unnecessary parameters, enhancing network performance.

Area of Science:

Artificial Intelligence
Machine Learning
Computational Neuroscience

Background:

Neural network complexity can hinder generalization, similar to overfitted regression functions.
Existing network pruning methods (e.g., Skeletonization, Optimal Brain Damage, Optimal Brain Surgeon) are primarily non-statistical.
Excessive parameters in neural networks can lead to poor performance on future data.

Purpose of the Study:

To develop a statistically rigorous method for pruning neural networks.
To improve the generalization capabilities of neural networks by reducing complexity.
To compare the proposed method against existing state-of-the-art pruning techniques.

Main Methods:

Utilizing the bootstrap algorithm to estimate the distribution of neural network parameter saliences.

Related Experiment Videos

Employing statistical multiple comparison procedures for informed pruning decisions.

Evaluating the method's effectiveness in pruning and its impact on network performance.

Main Results:

The proposed bootstrap-based statistical method effectively identifies and prunes less salient parameters.
The new method demonstrates comparable or superior performance to Optimal Brain Surgeon.
Pruned networks exhibit improved generalization ability on unseen data.

Conclusions:

A novel statistical approach using bootstrapping and multiple comparisons offers an effective strategy for neural network pruning.
This method provides a statistically sound alternative to existing non-statistical pruning techniques.
The approach enhances neural network generalization and performance while reducing model complexity.