Training two-layered feedforward networks with variable projection method

The variable projection (VP) method for separable nonlinear least squares (SNLLS) is presented and incorporated into the Levenberg-Marquardt optimization algorithm for training two-layered feedforward neural networks. It is shown that the Jacobian of variable projected networks can be computed by simple modification of the backpropagation algorithm. The suggested algorithm is efficient compared to conventional techniques such as conventional Levenberg-Marquardt algorithm (LMA), hybrid gradient algorithm (HGA), and extreme learning machine (ELM).