Related Experiment Videos
Efficient block training of multilayer perceptrons
A Navia-Vázquez1, A R Figueiras-Vidal
1DTC, Universidad Carlos III de Madrid, Spain.
Neural Computation
|August 10, 2000
Summary
This study refines layerwise block training for multilayer perceptrons (MLP) by adding a sensitivity correction factor, significantly improving performance. The enhanced method shows advantages over existing techniques in various applications.
Area of Science:
- Machine Learning
- Artificial Intelligence
- Neural Networks
Background:
- Layerwise block training offers computational advantages for multilayer perceptrons (MLP).
- Existing methods may lack optimal performance due to formulation limitations.
Purpose of the Study:
- To enhance layerwise block training algorithms for MLPs.
- To introduce a sensitivity correction factor for improved performance.
- To analyze the theoretical underpinnings of the performance gains.
Main Methods:
- Modification of layerwise block training algorithms.
- Introduction of a sensitivity correction factor.
- Empirical verification across several applications.
- Theoretical analysis relating to second-order methods and Fisher's information matrix.
Main Results:
- The refined algorithm demonstrates a clear performance advantage.
- The sensitivity correction factor is shown to be crucial for the performance gains.
- The method's effectiveness is validated through practical applications.
Conclusions:
- The proposed sensitivity correction factor significantly enhances MLP training.
- The approach offers a promising direction for improving neural network training efficiency and effectiveness.
- Potential extensions to recurrent networks and other research areas are identified.