Efficient IntVec: High recognition rate with reduced computational cost

In many deep neural networks for pattern recognition, the input pattern is classified in the deepest layer based on features extracted through intermediate layers. IntVec (interpolating-vector) is known to be a powerful method for this process of classification. Although the recognition error can be made much smaller by IntVec than by WTA (winner-take-all) or even by SVM (support vector machines), IntVec requires a large computational cost. This paper proposes a new method, by which the computational cost by IntVec can be reduced drastically without increasing the recognition error. Although we basically use IntVec for recognition, we substitute it with WTA, which requires much smaller computational cost, under a certain condition. To be more specific, we first try to classify the input vector using WTA. If a class is a complete loser by WTA, we judge it also a loser by IntVec and omit the calculation of IntVec for that class. If a class is an unrivaled winner by WTA, calculation of IntVec itself can be omitted for all classes.