Discriminant pattern recognition using transformation-invariant neurons

To overcome the problem of invariant pattern recognition, Simard, LeCun, and Denker (1993) proposed a successful nearest-neighbor approach based on tangent distance, attaining state-of-the-art accuracy. Since this approach needs great computational and memory effort, Hastie, Simard, and Säckinger (1995) proposed an algorithm (HSS) based on singular value decomposition (SVD), for the generation of nondiscriminant tangent models. In this article we propose a different approach, based on a gradient-descent constructive algorithm, called TD-Neuron, that develops discriminant models. We present as well comparative results of our constructive algorithm versus HSS and learning vector quantization (LVQ) algorithms. Specifically, we tested the HSS algorithm using both the original version based on the two-sided tangent distance and a new version based on the one-sided tangent distance. Empirical results over the NIST-3 database show that the TD-Neuron is superior to both SVD- and LVQ-based algorithms, since it reaches a better trade-off between error and rejection.