Inverting feedforward neural networks using linear and nonlinear programming

The problem of inverting trained feedforward neural networks is to find the inputs which yield a given output. In general, this problem is an ill-posed problem because the mapping from the output space to the input space is a one-to-many mapping. In this paper, we present a method for dealing with the inverse problem by using mathematical programming techniques. The principal idea behind the method is to formulate the inverse problem as a nonlinear programming (NLP) problem, a separable programming (SP) problem, or a linear programming (LP) problem according to the architectures of networks to be inverted or the types of network inversions to be computed. An important advantage of the method over the existing iterative inversion algorithm is that various designated network inversions of multilayer perceptrons (MLP's) and radial basis function (RBF) neural networks can be obtained by solving the corresponding SP problems, which can be solved by a modified simplex method, a well-developed and efficient method for solving LP problems. We present several examples to demonstrate the proposed method and the applications of network inversions to examining and improving the generalization performance of trained networks. The results show the effectiveness of the proposed method.