Features selection and architecture optimization in connectionist systems

In this paper, we propose a features selection measure and an architecture optimization procedure for Multi-Layer Perceptrons (MLP). The algorithm presented in this contribution employs a heuristic measure named HVS (Heuristic for Variable Selection). This new measure allows us to identify and select important variables in the features space. This can be achieved by eliminating redundant features and those which do not contain enough relevant information. The proposed measure is used in a new procedure aimed at selecting the "best" MLP architecture given an initial structure. Application results for two generic problems: regression and discrimination, demonstrates the proposed selection algorithm's effectiveness in identifying optimized connectionist models with higher accuracy. Finally, an extension of HVS, named epsilonHVS, is proposed for discriminative features detection and architecture optimization for Time Delay Neural Networks models (TDNN).