Distance Correlation-Based Feature Selection in Random Forest.

Area of Science:

• Statistics
• Machine Learning
• Data Mining

Background:

• Pearson correlation coefficient (ρ) is limited to linear relationships between variables.
• Existing feature selection methods may not capture complex, nonlinear dependencies.
• There is a need for methods that can identify all types of variable dependencies.

Purpose of the Study:

• To propose a novel filter method for feature selection in Random Forest regression.
• To utilize distance correlation as a criterion for identifying relevant features.
• To evaluate the proposed method's performance against existing techniques.

Main Methods:

• Implementing a filter method based on distance correlation for feature selection.
• Utilizing Random Forest regression as the predictive model.
• Conducting extensive simulation studies across various data settings.
• Comparing prediction mean squared error (MSE) with existing methods.

Main Results:

• The proposed distance correlation filter method is competitive with existing approaches.
• The method significantly outperforms other techniques in high-dimensional (p≥300) datasets with nonlinear relationships.
• The method demonstrates practical applicability through real-world data examples.

Conclusions:

• Distance correlation is a powerful tool for feature selection, capturing nonlinear dependencies.
• The proposed filter method enhances Random Forest regression performance, especially in high-dimensional, complex datasets.
• This approach offers a valuable advancement for statistical modeling and machine learning applications.