Nested ensemble selection: An effective hybrid feature selection method.

Area of Science:

• Machine Learning
• Data Science
• Bioinformatics

Background:

• Traditional feature selection methods struggle to distinguish between relevant, redundant, and correlated features.
• Existing hybrid filter-wrapper methods often lack simplicity, efficiency, or precision.

Purpose of the Study:

• To propose a novel feature selection algorithm, Nested Ensemble Selection (NES), that addresses limitations of existing methods.
• To enhance the precision and efficiency of feature selection by differentiating relevant from irrelevant, redundant, and correlated features.
• To introduce a heuristic for determining the optimal number of selected features.

Main Methods:

• Nested Ensemble Selection (NES), a hybrid approach combining filter and wrapper methods.
• Development of a robust heuristic for optimal feature subset identification.
• Comparative analysis against established algorithms like mRMR, Boruta, and genetic algorithms.

Main Results:

• NES achieves perfect precision on synthetic datasets and near-optimal accuracy on real-world data.
• The algorithm effectively separates relevant features from irrelevant, redundant, and correlated ones.
• NES demonstrates significant performance improvements over benchmark algorithms, especially on multi-class datasets.

Conclusions:

• Nested Ensemble Selection (NES) offers a simple, efficient, and precise solution for feature selection.
• The proposed method overcomes key challenges in feature selection, including identifying redundant and correlated features.
• NES represents a significant advancement in feature selection techniques, particularly for complex, multi-class data.