Heterogeneous Feature Selection Based on Neighborhood Combination Entropy.

Area of Science:

• Data Science
• Machine Learning
• Artificial Intelligence

Background:

• Feature selection is crucial for managing high-dimensional data, improving model performance, and enhancing interpretability.
• Heterogeneous data, containing both numerical and categorical attributes, are prevalent in real-world applications.
• Neighborhood rough set (NRS) is a powerful tool for handling heterogeneous data through neighborhood relations.

Purpose of the Study:

• To develop a unified feature selection framework for categorical, numerical, and heterogeneous data.
• To introduce neighborhood combination entropy (NCE) and conditional neighborhood combination entropy (cNCE) for feature evaluation.
• To design and validate a novel feature selection algorithm, FScNCE, based on cNCE.

Main Methods:

• The study presents neighborhood combination entropy (NCE) to measure distinguishability of neighborhood granules.
• Conditional neighborhood combination entropy (cNCE) is proposed, considering decision attributes.
• A feature selection algorithm (FScNCE) is developed using inner and outer significance functions based on cNCE.

Main Results:

• The proposed FScNCE algorithm demonstrates effectiveness in feature selection for heterogeneous datasets.
• Experimental results validate the superiority of the FScNCE algorithm compared to existing methods.
• The method successfully alleviates the dimensionality curse and enhances learning performance.

Conclusions:

• The developed FScNCE algorithm provides an effective approach for feature selection on heterogeneous data.
• The novel cNCE measure offers a robust way to evaluate feature relevance in mixed-attribute environments.
• This research contributes to advancing feature selection techniques for complex, real-world datasets.