Unbiased feature selection in learning random forests for high-dimensional data.

Area of Science:

• Machine Learning
• Data Science
• Computer Science

Background:

• Random Forests (RFs) are popular for classification but struggle with high-dimensional data due to uninformative feature selection.
• RFs exhibit bias favoring multi-valued features, impacting performance.
• The randomization in RFs can lead to suboptimal feature selection for node splitting.

Purpose of the Study:

• To propose a novel RF algorithm, xRF, for improved feature selection in high-dimensional datasets.
• To enhance the accuracy and debias the feature selection process of RFs.
• To reduce dimensionality and data requirements for learning RFs.

Main Methods:

• Uninformative features are identified and removed using p-value assessment.
• A subset of unbiased features is selected based on statistical measures.
• Feature weighting sampling is applied to partitioned feature subsets for tree construction.

Main Results:

• The proposed xRF algorithm demonstrated superior performance over existing RF methods.
• Experiments on 47 high-dimensional datasets showed increased accuracy and AUC measures.
• The approach effectively handles high-dimensional data, including image datasets.

Conclusions:

• xRF offers a robust solution for debiasing feature selection in random forests.
• The algorithm enhances classification accuracy and AUC for high-dimensional data.
• xRF provides a method for more efficient learning by reducing dimensionality and data needs.