The convergence of backpropagation trained neural networks for various weight update frequencies

One of the problems concerning the backpropagation training of feed-forward neural networks is the effect of the weight update frequency. This aspect influences the efficiency of parallel implementations of the training algorithm where the training vectors are distributed among processors. In this paper the convergence of two applications for various weight update intervals is reported. Further, several models are proposed for describing convergence and learning rate aspects in the context of a set of weight update intervals. The results show that the convergence by updating the weights after each training vector leads to about 10 times less number of training iterations compared to updating the weights only ones for the whole training set.