Evolutionary Sparsity Regularisation-Based Feature Selection for Binary Classification.

Area of Science:

• Machine Learning
• Data Mining
• Computational Intelligence

Background:

• Feature selection is crucial for improving classification performance by reducing dimensionality.
• Embedded feature selection methods, like sparsity regularization, offer a good balance between performance and computation time.
• Existing methods often output feature rankings, requiring predefined subset sizes and risking the selection of redundant features due to ignored interactions.

Purpose of the Study:

• To propose a new feature selection representation that accounts for feature interactions.
• To develop an algorithm that automatically determines the optimal number of features for classification.
• To enhance classification performance and efficiency over existing feature selection techniques.

Main Methods:

• A novel representation considering feature interactions was developed.
• A differential evolutionary (DE) algorithm was employed to optimize feature subsets.
• A unique initialization mechanism was introduced to enable DE to explore various feature subset sizes.

Main Results:

• The proposed algorithm successfully selected complementary features on synthetic datasets, avoiding redundancy issues common in sparsity regularization methods.
• On real-world datasets, the algorithm demonstrated superior classification performance compared to established wrapper, filter, and embedded approaches.
• The method achieved computational efficiency comparable to filter-based feature selection techniques.

Conclusions:

• The proposed feature selection approach effectively handles feature interactions and automatically determines subset size.
• This method offers improved classification accuracy and efficiency, outperforming existing techniques.
• The algorithm presents a significant advancement in automated and interaction-aware feature selection for classification tasks.