Splitting on categorical predictors in random forests.

Area of Science:

• Machine Learning
• Computational Statistics
• Data Mining

Background:

• Random Forests (RFs) are successful due to minimal data preprocessing requirements.
• Standard RF methods struggle with nominal predictors, requiring computationally expensive 2-partitions.
• Existing methods for nominal predictors increase complexity and limit category numbers.

Purpose of the Study:

• To develop an efficient method for handling nominal predictors in Random Forests.
• To reduce computational complexity and overcome category limitations in RFs.
• To improve the performance and efficiency of RFs for multiclass classification and survival prediction.

Main Methods:

• Proposed a heuristic to order nominal predictor categories using principal component analysis or log-rank scores.
• Categories can be ordered in each split or a priori before growing the forest.
• Compared the proposed ordering method against standard 2-partitions, dummy coding, and ignoring nominal nature.

Main Results:

• Ordering nominal predictor categories a priori is computationally faster than the standard approach.
• The proposed method achieves comparable or better prediction performance across various datasets.
• The heuristic ordering effectively treats nominal predictors as ordinal, speeding up computation.

Conclusions:

• A priori ordering of nominal predictor categories is a computationally efficient and effective enhancement for Random Forests.
• This method matches the performance of standard approaches while significantly reducing computational load.
• Recommending the a priori ordering approach as the default method for Random Forests.