Assessing the accuracy and stability of variable selection methods for random forest modeling in ecology

Area of Science:

• Ecology
• Environmental Science
• Statistical Modeling

Background:

• Random Forest (RF) modeling is a powerful tool in ecology for prediction.
• Limited guidance exists for variable selection in RF models with large ecological datasets.
• Common approaches include preselecting variables or using stepwise elimination based on importance measures.

Purpose of the Study:

• Investigate variable selection methods for RF models predicting biological stream condition.
• Compare RF models using full vs. reduced variable sets.
• Assess the impact of variable selection on model accuracy and spatial prediction stability.

Main Methods:

• Utilized data from the National Rivers and Stream Assessment (1365 sites) and StreamCat dataset (212 landscape features).
• Compared a full RF model (212 predictors) with a reduced model (backward elimination).
• Assessed accuracy using out-of-bag estimates and external cross-validation; evaluated spatial prediction stability.

Main Results:

• RF models are robust to including numerous variables of moderate to low importance.
• Variable reduction via backward elimination did not substantially improve cross-validated accuracy.
• Backward elimination led to selecting too few variables, biased accuracy estimates, and unstable spatial predictions.

Conclusions:

• Model selection for RF in ecology should balance accuracy and stability.
• Backward elimination is not recommended for RF variable selection in this context due to bias and instability.
• Ecologists should be aware of potential model selection biases when using RF with large environmental datasets.