Handwritten Digit Recognition Using K Nearest-Neighbor, Radial-Basis Function, and Backpropagation Neural Networks

Area of Science:

• Computer Science
• Machine Learning
• Pattern Recognition

Background:

• Multilayer neural networks with specific architectures are believed to excel at handwritten digit recognition.
• Previous research suggests unique performance advantages for certain network designs.

Purpose of the Study:

• To compare the performance of backpropagation networks, radial basis function (RBF) networks, and k-nearest-neighbor (kNN) classifiers on a large handwritten digit database.
• To evaluate practical constraints such as training time, memory usage, and classification time for each classifier.

Main Methods:

• Performance evaluation of backpropagation networks, RBF networks, and kNN classifiers.
• Analysis of error rates, training duration, memory requirements, and classification speed.
• Assessment of confidence judgments for rejecting ambiguous inputs.

Main Results:

• All three classifiers (backpropagation, RBF, kNN) demonstrated similar low error rates on the handwritten digit database.
• Backpropagation networks offered superior memory usage and classification time but longer training times and potential for false positives.
• RBF networks required more memory and classification time but less training time, with effective confidence judgments for ambiguous inputs.
• kNN classifiers, while simple and fast to train, incurred prohibitive memory costs and slow classification.

Conclusions:

• Practical considerations like training time, memory, and classification speed often outweigh minor differences in error rates when selecting a classifier.
• RBF classifiers provide robust confidence judgments for improved accuracy in critical applications.
• kNN classifiers are suitable for hardware-assisted tasks due to their simplicity and fast training.