Standard errors and confidence intervals for variable importance in random forest regression, classification, and...

Area of Science:

• Machine Learning
• Statistical Modeling
• Data Science

Background:

• Random forests are widely used for prediction and provide a nonparametric measure of variable importance (VIMP).
• A key limitation of VIMP is the lack of systematic methods for variance estimation.
• This hinders the construction of reliable confidence intervals for VIMP.

Purpose of the Study:

• To propose a novel subsampling approach for estimating VIMP variance.
• To enable the construction of confidence intervals for VIMP in random forests.
• To offer a computationally efficient solution applicable to large datasets.

Main Methods:

• A subsampling strategy is developed to estimate the variance of VIMP.
• The method is validated through extensive simulations across various settings (regression, classification, survival).
• Comparison with existing methods like the delete-d jackknife and bootstrap estimators is performed.

Main Results:

• The proposed subsampling method effectively estimates VIMP variance and facilitates confidence interval construction.
• The delete-d jackknife variance estimator shows particular efficacy at low subsampling rates due to bias correction.
• Subsampling-based estimators demonstrate competitive performance against bootstrap methods, especially in handling ties.

Conclusions:

• Subsampling offers a fast and effective solution for VIMP variance estimation in random forests.
• This method is broadly applicable across diverse data analysis problems, including big data scenarios.
• The proposed technique enhances the interpretability and reliability of VIMP in machine learning models.