The k conditional nearest neighbor algorithm for classification and class probability estimation.

Area of Science:

• Machine Learning
• Pattern Recognition
• Statistical Classification

Background:

• The k-nearest neighbors (kNN) algorithm is a widely used nonparametric method for classification.
• A key limitation of kNN is its inability to provide precise class probability estimates, especially with small sample sizes (low k values).

Purpose of the Study:

• To introduce a novel nonparametric classification method that enhances the probability estimation capabilities of kNN.
• To evaluate the performance of the proposed method and its ensemble extension against existing algorithms.

Main Methods:

• The proposed method calculates distances to the k-th nearest neighbor within each class.
• It estimates posterior probabilities based on these distances and assigns instances to the class with the highest probability.
• The approach is extended to an ensemble method for potentially improved robustness.

Main Results:

• The proposed method converges to the Bayes classifier as training data size increases.
• Experiments show the proposed method and its ensemble version outperform kNN, weighted kNN, probabilistic kNN, LMKNN, and MLM-kHNN in error rate.
• Simulations suggest utility in estimating posterior probabilities for overlapping class distributions.

Conclusions:

• The novel kNN-based classification method offers improved accuracy and probability estimation.
• The ensemble extension further enhances performance, providing a valuable alternative for classification tasks.
• This approach shows promise for scenarios with overlapping class distributions.