k-NS: a classifier by the distance to the nearest subspace

Area of Science:

• Machine Learning
• Pattern Recognition
• Data Mining

Background:

• The k-Nearest Neighbors (k-NN) algorithm is a fundamental classification method.
• Improving k-NN's classification performance, accuracy, and efficiency remains an active research area.
• Subspace-based methods offer potential for enhanced classification by considering local data geometry.

Purpose of the Study:

• To propose a novel classifier, k-NS (k-Nearest Subspaces), designed to enhance classification performance.
• To leverage Euclidean distances to nearest class-specific subspaces for improved sample categorization.
• To extend the classifier's applicability to high-dimensional feature spaces using kernel functions.

Main Methods:

• The k-NS classifier calculates distances from a query sample to nearest subspaces.
• Each subspace is spanned by the k nearest samples belonging to the same class.
• A discriminant based on the Grammian, stabilized by Tikhonov regularization, is used for distance calculation.
• The classifier naturally extends to kernel-induced feature spaces via inner products.

Main Results:

• Experimental results on 13 benchmark datasets demonstrate k-NS's effectiveness.
• k-NS shows promising improvements in both training and test accuracy compared to other k-NN-based classifiers.
• The proposed method also exhibits enhanced computational efficiency.

Conclusions:

• The k-NS classifier provides a robust and efficient approach for improving classification performance.
• Its subspace-based methodology and kernel extension capabilities make it a valuable tool in machine learning.
• k-NS represents a promising advancement over traditional nearest neighbor algorithms.