Iterative fast orthogonal search algorithm for MDL-based training of generalized single-layer networks

The generalized single-layer network (GSLN) architecture, which implements a sum of arbitrary basis functions defined on its inputs, is potentially a flexible and efficient structure for approximating arbitrary nonlinear functions. A drawback of GSLNs is that a large number of weights and basis functions may be required to provide satisfactory approximations. In this paper, we present a new approach in which an algorithm known as iterative fast orthogonal search (IFOS) is coupled with the minimum description length (MDL) criterion to provide automatic structure selection and parameter estimation for GSLNs. The resulting algorithm, dubbed IFOS-MDL, performs both network growth and pruning to construct sparse GSLNs from potentially large spaces of candidate basis functions.