optRF: Optimising random forest stability by determining the optimal number of trees

Area of Science:

• Machine Learning
• Data Science
• Computational Statistics

Background:

• Random forest is a prominent machine learning algorithm for big data analysis.
• A key characteristic of random forest is its non-deterministic nature, potentially leading to varied model outputs from identical datasets.
• This variability can significantly impact decision-making processes reliant on these models.

Purpose of the Study:

• To develop a method for quantifying the impact of non-determinism in random forest models.
• To assess how non-determinism affects predictions, variable importance estimates, and subsequent decisions.
• To identify an optimal number of trees for random forest models to balance stability and computational cost.

Main Methods:

• Introduced a novel method to quantify the non-determinism of random forest models.
• Analyzed the relationship between the number of trees and model stability.
• Evaluated the impact on predictions and variable importance estimates.
• Developed the R package optRF to model the tree-number-stability relationship.

Main Results:

• Increasing the number of trees in random forests improves model stability in a non-linear fashion.
• Computational time increases linearly with the number of trees.
• An optimal number of trees exists that maximizes stability without excessive computational overhead.

Conclusions:

• Non-determinism in random forests necessitates careful consideration for reliable decision-making.
• The R package optRF provides data-driven recommendations for optimal tree numbers.
• Balancing model stability and computational efficiency is achievable through optimized tree selection.