Feature Selection Based on Neighborhood Discrimination Index.

Area of Science:

• Computer Science
• Machine Learning
• Data Mining

Background:

• Feature selection is crucial for optimizing pattern recognition and machine learning models.
• Neighborhood concepts are vital for distinguishing samples in classification tasks.
• Existing methods may not fully capture the distinguishing information within feature subsets.

Purpose of the Study:

• To propose a novel neighborhood discrimination index for quantifying the distinguishing information of feature subsets.
• To introduce variants of the index (joint, conditional, mutual) for analyzing combinations of feature subsets.
• To develop a feature selection algorithm based on these discrimination measures.

Main Methods:

• A neighborhood discrimination index is proposed, focusing on neighborhood relation cardinality.
• Variants like joint, conditional, and mutual discrimination indices are introduced.
• A neighborhood radius parameter is incorporated for real-valued data analysis.
• A greedy forward algorithm for feature selection is designed using the proposed measures.

Main Results:

• The proposed neighborhood discrimination index effectively reflects the distinguishing ability of feature subsets.
• Variants of the index exhibit properties similar to Shannon entropy.
• The developed feature selection algorithm demonstrates superior performance compared to classical algorithms in experiments.
• Experimental validation on public datasets confirms the algorithm's effectiveness.

Conclusions:

• The neighborhood discrimination index offers a robust measure for feature selection.
• The proposed algorithm provides a significant improvement in classification performance.
• This approach enhances the preprocessing step in machine learning and data mining pipelines.