Post model-fitting exploration via a "Next-Door" analysis.

Area of Science:

• Statistics
• Machine Learning
• Computational Statistics

Background:

• Adaptive regression procedures, particularly the lasso, are widely used for model selection.
• Evaluating the stability and contribution of predictors in selected models is crucial for reliable analysis.
• Existing methods may not fully capture the nuances of predictor importance and alternative model selection.

Purpose of the Study:

• To propose a novel method, "Next-Door analysis," for evaluating models selected by adaptive regression procedures.
• To assess the indispensability of predictors within a chosen model.
• To identify acceptable alternative models that offer comparable predictive performance.

Main Methods:

• The core method involves systematically deleting each predictor from the chosen "base model."
• Refitting the lasso (or other penalized/stepwise procedures) to generate "nearby" models.
• Comparing the error rates of the base model with those of the nearby models to assess predictor impact.

Main Results:

• Predictors whose deletion significantly deteriorates model performance are identified as "indispensable."
• Models resulting from the deletion of non-indispensable predictors are deemed "acceptable alternatives."
• The analysis provides both a quantitative assessment of each variable's contribution and a set of alternative models.

Conclusions:

• Next-Door analysis offers a robust framework for model evaluation in the context of adaptive regression.
• The method enhances interpretability by quantifying predictor importance and providing alternative model choices.
• Implementation in R, compatible with glmnet, makes this technique accessible for practical applications in supervised learning with L1 penalization.