Minimizing features while maintaining performance in data classification problems.

Area of Science:

• Machine Learning
• Data Science

Background:

• High-dimensional classification is a growing challenge in machine learning.
• Effective feature selection is crucial for optimizing machine learning algorithms.
• Existing methods often select the highest-scoring feature subset, potentially missing smaller, near-optimal subsets.

Purpose of the Study:

• To propose and apply an extended feature selection method (PCLFS-ext).
• To enable the selection of smaller feature subsets with performance similar to larger subsets within a defined threshold.
• To validate the efficacy of PCLFS-ext through simulations.

Main Methods:

• An extended version of Principal Component Loading Feature Selection (PCLFS-ext) was developed.
• Simulated data with varying numbers of features and imbalance rates were used for validation.
• Performance was evaluated across several classification methods.

Main Results:

• The proposed PCLFS-ext method demonstrated superior performance compared to the original PCLFS and Recursive Feature Elimination (RFE).
• PCLFS-ext achieved significant and reasonable feature reduction across diverse datasets.
• The method maintained comparable performance with reduced feature sets.

Conclusions:

• The extended Principal Component Loading Feature Selection (PCLFS-ext) method effectively reduces features while maintaining classification performance.
• This approach is valuable for applications where feature reduction is critical.
• PCLFS-ext offers an improved alternative to existing feature selection techniques like RFE.