Formal Hypothesis Tests for Additive Structure in Random Forests.

Area of Science:

• Statistics
• Machine Learning
• Computational Statistics

Background:

• Statistical learning models offer powerful predictive capabilities but often lack interpretability due to their 'black-box' nature.
• Ensemble learners, such as random forests, possess inherent structures that can be leveraged for statistical inference.
• Existing methods struggle to provide formal hypothesis testing for complex models like random forests.

Purpose of the Study:

• To develop formal statistical tests for variable importance and additive model structure within random forests.
• To enhance the interpretability and inferential capabilities of ensemble learning methods.
• To introduce computationally efficient testing procedures for large-scale datasets.

Main Methods:

• Defining a grid structure on the covariate space to enable hypothesis testing.
• Developing notions of total and partial additivity.
• Estimating variance during ensemble construction for cost-free testing.
• Utilizing random projections for computationally efficient extensions.

Main Results:

• Demonstrated the feasibility of formal hypothesis tests for variable importance and additive structure in random forests.
• Showcased that testing can be performed at no additional computational cost.
• Proposed a novel extension using random projections for efficient testing on large grids.

Conclusions:

• These tests are the first of their kind for investigating regression structure in random forests.
• The proposed methods significantly improve the interpretability and inferential power of random forests.
• The random projection extension ensures computational efficiency and high power even with large grid sizes.