Alternative stopping rules to limit tree expansion for random forest models.

Area of Science:

• Machine Learning
• Statistics
• Data Science

Background:

• Random forests are robust machine learning models adept at capturing non-linear relationships.
• Standard random forest models have limited hyperparameters, including minimum terminal node size.
• Existing stopping rules limit parent node size, while newer rules ensure minimum observations in terminal nodes.

Purpose of the Study:

• To propose and evaluate three generalized stopping criteria for regression random forests.
• To introduce new methods based on variance, range, and inter-centile range for controlling tree growth.
• To assess the performance of these novel stopping rules against standard random forest models.

Main Methods:

• Developed three generalized stopping criteria for regression random forests: variance, range, and inter-centile range.
• Applied the proposed methods to diverse datasets, including the National Health and Nutrition Examination Survey (diabetes data).
• Compared the mean square prediction error (MSPE) of the new rules against standard random forest models.

Main Results:

• The proposed stopping rules achieved competitive mean square prediction error (MSPE) compared to standard random forest models.
• Using the inter-centile range statistic resulted in significantly less variation in MSPE.
• The inter-centile range method also produced MSPE values closer to the optimal.

Conclusions:

• The novel stopping rules offer a viable alternative for regression random forests, demonstrating competitive performance.
• The inter-centile range stopping criterion is particularly effective in reducing MSPE variation and achieving near-optimal results.
• The study provides valuable insights into optimizing random forest model performance through advanced stopping criteria.