Feature selection based on distance correlation: a filter algorithm.

Area of Science:

• Machine Learning
• Data Science
• Statistical Modeling

Background:

• The curse of dimensionality poses a significant challenge in machine learning and data analysis.
• Feature selection (FS) is a critical technique for mitigating dimensionality issues and improving model performance.
• Existing FS methods often require parameter tuning or model-specific assumptions.

Purpose of the Study:

• To propose a novel, model-free filter approach for feature selection.
• To introduce a new method, Distance Correlation Feature Selection (DCFS), that leverages distance correlation.
• To address the limitations of existing feature selection techniques by eliminating the need for pre-specified parameters.

Main Methods:

• Developed a two-step feature selection process: a hard step (forward selection) and a soft step (backward selection).
• The hard step identifies relevant features by examining associations between features and target classes.
• The soft step refines feature selection using a feature-relationship gain based on distance correlation, considering five types of associations.

Main Results:

• The proposed DCFS method demonstrated competitive performance across various datasets.
• Simulation results indicate that DCFS outperforms several representative feature selection methods.
• The model-free nature and lack of pre-specified parameters enhance its applicability.

Conclusions:

• Distance Correlation Feature Selection (DCFS) offers an effective and robust approach to feature selection.
• The method successfully addresses the curse of dimensionality while maintaining a model-free advantage.
• DCFS provides a competitive alternative to existing feature selection techniques in machine learning applications.