HeFS: helper-enhanced feature selection via Pareto-optimized genetic search

Feature selection plays a crucial role in improving predictive performance and interpretability in high-dimensional machine learning tasks. However, it is an NP-hard combinatorial optimization problem. Conventional heuristic or greedy approaches are prone to premature convergence and often yield locally optimal feature subsets, especially in high-dimensional settings with complex feature interactions.

This study aims to enhance existing feature selection methods by identifying complementary helper features overlooked by conventional approaches through a plug-and-play refinement framework.

We formulate Conditional Feature Selection as augmenting an existing feature subset by discovering complementary helper features from the residual feature space, i.e., features typically discarded by conventional selection methods. Helper-Enhanced Feature Selection (HeFS) implements this idea as a post-selection refinement framework that can be integrated with diverse baseline algorithms. By incorporating biased initialization and ratio-guided mutation and employing a multi-objective search strategy that balances predictive accuracy and feature complementarity, HeFS enables the systematic identification of informative yet previously overlooked features.

A comprehensive evaluation on 18 benchmark datasets shows that HeFS consistently enhances baseline feature selection performance by identifying informative helper features overlooked by conventional methods. On average, the helper features identified by HeFS improve classification accuracy by more than 9% over traditional feature selection methods and 6% over state-of-the-art approaches, with the largest gain observed on the Spambase dataset, where accuracy increases by 31.8% over the MGWO baseline. Further analysis on molecular datasets using Pearson correlation coefficients indicates that helper features exhibit substantially lower correlation with baseline-selected features (0.06) than the correlations among baseline features themselves (0.24), suggesting reduced redundancy and complementary contributions.

By explicitly modeling feature complementarity in the residual feature space, HeFS mitigates the limitations of conventional heuristic or greedy feature selection methods and improves classification performance across diverse application domains.