Related Experiment Videos
Analytical interpretation of feed-forward nets outputs after training
1Department d'Estructura i Constituents de la Matèria, Facultat de Física, Universitat de Barcelona, Spain.
International Journal of Neural Systems
|March 1, 1996
Summary
This study uses functional analysis to examine the quadratic error criterion and back-propagation algorithm. It reveals that optimal classification requires specific output representations for the Bayesian decision rule.
Area of Science:
- Machine Learning
- Functional Analysis
- Statistical Learning Theory
Background:
- The back-propagation algorithm is a cornerstone of modern machine learning, widely used for training artificial neural networks.
- Minimizing quadratic error is a common objective function, but its theoretical underpinnings, especially concerning optimal decision rules, warrant deeper investigation.
- Understanding the relationship between error criteria and resulting decision rules is crucial for developing more robust and efficient learning algorithms.
Purpose of the Study:
- To analyze the minimization quadratic error criterion and its connection to the back-propagation algorithm using functional analysis.
- To theoretically derive the nature of the global minimum for quadratic error minimization.
- To investigate the implications for classification tasks and explore alternative error criteria.
Main Methods:
- Application of functional analysis techniques to study the quadratic error criterion.
- Derivation of the statistical properties of the global minimum of the error function.
- Analysis of classification tasks under different output class representations.
- Exploration of alternative error metrics, such as absolute value errors.
Main Results:
- The global minimum of the quadratic error criterion corresponds to a function that outputs the expected value of the target variable for each input pattern.
- For classification, only specific output class representations allow for the derivation of the optimal Bayesian decision rule.
- The study demonstrates that minimizing absolute value errors leads to predicting medians instead of mean values.
Conclusions:
- Functional analysis provides a rigorous framework for understanding the back-propagation algorithm and error minimization.
- The choice of output representation is critical for achieving optimal Bayesian classification when using quadratic error.
- Alternative error criteria, like absolute value error, yield different statistical estimators (medians vs. means), highlighting the importance of criterion selection.