Related Experiment Videos
Discriminant component pruning. Regularization and interpretation of multi-layered back-propagation networks
1Department of Psychology, McGill University, Montreal, Quebec H3A 1B1, Canada. randalk@marina.psych.mcgill.ca
Neural Computation
|March 23, 1999
Summary
Discriminant Components Pruning (DCP) reduces neural network complexity while maintaining performance. This method aids in interpreting learned functions and improving generalization for classification tasks.
Area of Science:
- Computer Science
- Artificial Intelligence
- Machine Learning
Background:
- Neural networks are widely used for classification tasks.
- Large neural networks can increase complexity, hindering interpretation and potentially reducing generalization.
- Network pruning is a technique to manage complexity.
Purpose of the Study:
- To introduce Discriminant Components Pruning (DCP), a novel method for pruning neural networks.
- To demonstrate DCP's effectiveness in reducing network complexity.
- To show DCP's utility in enhancing network interpretability and maintaining generalization performance.
Main Methods:
- DCP prunes matrices of summed contributions between neural network layers.
- The method focuses on reducing the rank of these matrices.
- The study evaluates DCP's impact on generalization and interpretability.
Main Results:
- DCP effectively reduces neural network complexity.
- The pruning method maintains optimal generalization performance.
- DCP proves useful for interpreting the functions learned by neural networks.
Conclusions:
- DCP is an effective technique for simplifying neural networks without sacrificing performance.
- The method offers a valuable tool for network interpretation.
- Future work may involve optimizing rank identification and incorporating nonlinear activation functions.