Related Experiment Videos
Dynamic tunneling technique for efficient training of multilayer perceptrons
P RoyChowdhury1, Y P Singh, R A Chansarkar
1Defence Terrain Research Laboratory, Delhi, India.
IEEE Transactions on Neural Networks
|February 7, 2008
Summary
A novel training method for multilayer feedforward neural networks uses gradient descent and dynamic tunneling to avoid local traps, ensuring global minimum convergence. This efficient computational technique overcomes initialization challenges in neural network training.
Area of Science:
- Computational neuroscience
- Machine learning algorithms
- Artificial intelligence
Background:
- Multilayer feedforward neural networks are widely used but susceptible to local minima during training.
- Initialization sensitivity and local optima pose significant challenges in achieving global convergence.
Purpose of the Study:
- To propose a new, efficient computational technique for training multilayer feedforward neural networks.
- To overcome the limitations of local minima and initialization dependency in neural network training.
Main Methods:
- The proposed algorithm integrates two learning phases: gradient descent for local search and dynamic tunneling for direct search in weight space.
- Alternating these phases enables the search to escape local traps and identify points for further descent.
- The procedure is repeated to achieve convergence to a global minimum.
Main Results:
- Simulation results on five test examples demonstrate the method's efficiency.
- The technique successfully avoids local minima, leading to a global minimum solution.
- The approach is robust to arbitrary initial weight choices.
Conclusions:
- The proposed training procedure offers an efficient and effective solution for multilayer perceptron training.
- It successfully addresses the critical issues of initialization and local minima.
- This method enhances the reliability and performance of neural network models.