Related Experiment Videos
Efficient training algorithms for a class of shunting inhibitory convolutional neural networks
Fok Hing Chi Tivive1, Abdesselam Bouzerdoum
1School of Electrical, Computer, and Telecommunications Engineering, University of Wollongong, Wollongong, NSW 2522, Australia. fhct243@uow.edu.au
IEEE Transactions on Neural Networks
|June 9, 2005
Summary
New training algorithms for shunting inhibitory convolutional neural networks (CoNNs) offer efficient face recognition. A novel hybrid method matches Levenberg-Marquardt performance with reduced computational cost and memory needs.
Area of Science:
- Artificial Intelligence
- Machine Learning
- Computer Vision
Background:
- Convolutional Neural Networks (CoNNs) are powerful tools for pattern recognition.
- Shunting inhibitory CoNNs require efficient training algorithms for optimal performance.
- Existing training methods may be computationally intensive or memory-demanding.
Purpose of the Study:
- To develop and evaluate efficient training algorithms for shunting inhibitory CoNNs.
- To introduce a novel hybrid training method combining established optimization principles.
- To assess the performance of various training algorithms on a face/nonface discrimination task.
Main Methods:
- Implementation of 16 training algorithms, including first-order, second-order, conjugate gradient, Quickprop, Rprop, SuperSAB, and least squares (LS).
- Development of a new hybrid training method integrating principles from Quickprop, Rprop, SuperSAB, and LS.
- Evaluation using face/nonface discrimination for three CoNN architectures: binary, Toeplitz, and fully connected.
- Assessment of classification accuracy via ten-fold cross-validation.
Main Results:
- The new hybrid method demonstrates comparable performance to the Levenberg-Marquardt (LM) algorithm with lower computational cost and memory usage.
- Combinations of LS with the new hybrid method and LS with LM achieved the fastest convergence rates.
- Toeplitz-connected and binary-connected CoNN architectures slightly outperformed the fully connected architecture, with error rates of 1.95% and 2.10%, respectively.
- Modified BFGS, LM variants, and the new hybrid/LS method consistently achieved error rates below 3% across all architectures.
Conclusions:
- The proposed hybrid training method offers an efficient alternative for training shunting inhibitory CoNNs.
- Optimized training algorithms significantly impact convergence speed and classification accuracy in CoNNs.
- Specialized CoNN architectures (Toeplitz and binary) show potential for improved performance in visual pattern recognition tasks.