Related Experiment Videos
Assessing the importance of features for multi-layer perceptrons
Michael Egmont-Petersen1, Jan L. Talmon, Arie Hasman
1Department of Medical Informatics, Maastricht University, Maastricht, The Netherlands
Summary
This study introduces a mathematical framework to measure feature importance in classifiers. Metrics like replaceability and expected influence accurately rank features and aid in network pruning, improving model performance.
Area of Science:
- Machine Learning
- Artificial Intelligence
- Computational Neuroscience
Background:
- Assessing feature importance is crucial for understanding and optimizing classifier performance.
- Existing methods may not accurately quantify individual feature contributions to complex models like neural networks.
Purpose of the Study:
- To develop a mathematical framework for quantifying feature contribution to classifier performance.
- To design metrics for estimating feature importance in multi-layer perceptron neural networks.
- To introduce a method for pruning input nodes while preserving network knowledge.
Main Methods:
- Establishing a mathematical framework for feature contribution measures.
- Designing specific metrics (e.g., replaceability, expected influence) for feature importance estimation.
- Developing and validating an input node pruning technique for neural networks.
Main Results:
- All developed measures provide lower bounds on classifier correctness when features are excluded.
- The 'replaceability' metric yields the tightest error bounds.
- Both 'replaceability' and 'expected influence' metrics effectively rank feature importance.
Conclusions:
- The proposed framework and metrics offer robust methods for feature analysis in classifiers.
- The pruning method effectively retains network knowledge while simplifying the model.
- These techniques are valuable for interpreting and enhancing neural network performance.