A Very Simple Safe-Bayesian Random Forest.

Area of Science:

• Machine Learning
• Computational Statistics

Background:

• Random forests are ensemble methods that average predictions from de-correlated decision trees.
• Existing methods often rely on computationally intensive or potentially inaccurate probabilistic models.

Purpose of the Study:

• To introduce a new, efficient, and robust random forest generation method.
• To develop a 'Safe-Bayesian' approach that ensures good predictive performance.

Main Methods:

• Randomly sampling decision trees from a prior distribution.
• Employing a weighted ensemble of predictive probabilities using a power likelihood for aggregation.
• Developing a 'safe' aggregation procedure that is not strictly Bayesian but offers robustness.

Main Results:

• The proposed Safe-Bayesian random forest demonstrates superior speed and accuracy compared to Markov Chain Monte Carlo (MCMC) or Sequential Monte Carlo (SMC) based Bayesian decision trees.
• It achieves performance competitive with traditional entropy or Gini optimized random forests.
• The method is empirically shown to be simple to construct.

Conclusions:

• The Safe-Bayesian random forest offers a computationally efficient and accurate alternative for ensemble modeling.
• Its 'safeness' ensures reliable predictions, making it a valuable tool even when underlying assumptions are violated.
• This approach simplifies the construction of powerful random forests.