Related Experiment Videos
Aitken-based acceleration methods for assessing convergence of multilayer neural networks
R S Pilla1, S V Kamarthi, B G Lindsay
1Division of Epidemiology and Biostatistics, University of Illinois, Chicago, IL 60612, USA. pillar@uic.edu
IEEE Transactions on Neural Networks
|February 6, 2008
Summary
This study introduces an advanced Aitken acceleration method to speed up neural network training. The new approach effectively predicts the final error, offering a reliable stopping criterion for achieving desired accuracy.
Area of Science:
- Artificial Intelligence
- Machine Learning
- Numerical Analysis
Background:
- Neural network training with backpropagation can yield slow-converging error sequences.
- The standard Aitken delta(2) method is effective for geometric error sequences but suboptimal for log-geometric ones.
Purpose of the Study:
- To develop an acceleration method for log-geometric error sequences in neural network training.
- To enable accurate prediction of the final error value for improved convergence assessment.
Main Methods:
- Extension of the Aitken delta(2) method to an invariant extended-Aitken acceleration approach.
- Application to error sequences generated by neural networks trained with sigmoidal activation functions via backpropagation.
Main Results:
- The invariant extended-Aitken method effectively accelerates log-geometric sequences.
- The method provides outstanding prediction of the final error, outperforming standard techniques.
Conclusions:
- The developed acceleration method offers improved convergence rates for neural network training.
- The error prediction capability serves as a robust stopping criterion for achieving desired accuracy.