Identifying (Quasi) Equally Informative Subsets in Feature Selection Problems for Classification: A Max-Relevance...

Area of Science:

• Machine Learning
• Data Mining
• Bioinformatics

Background:

• Traditional feature selection often focuses on optimizing a single criterion, like accuracy, or a simple tradeoff.
• Existing methods may overlook multiple feature subsets with similar predictive power for a given number of features.

Purpose of the Study:

• To develop a multiobjective feature selection approach that identifies a performance-maximizing subset and quasi-equally informative subsets.
• To enable a deeper analysis of feature selection tradeoffs and feature importance.

Main Methods:

• Introduced a wrapper-based approach (W-QEISS) for quasi-equally informative subset selection.
• Formulated a four-objective optimization problem: maximizing classifier accuracy, minimizing feature count, and optimizing entropy-based relevance and redundancy.
• Utilized the Borg Multiobjective Evolutionary Algorithm and Extreme Learning Machine.

Main Results:

• The W-QEISS approach generated a rich and diverse set of Pareto-efficient solutions across 24 UCI datasets.
• Compared favorably against mRMR, a two-objective wrapper, and an F-QEISS filter.
• Demonstrated the utility of quasi-equally informative subsets for analyzing performance tradeoffs and feature importance.

Conclusions:

• The proposed W-QEISS method effectively identifies diverse, high-performing feature subsets.
• This approach provides valuable insights into the complex relationships between feature selection, classifier performance, and feature relevance.
• The method offers a more comprehensive understanding of feature selection compared to traditional two-objective algorithms.