Model confidence bounds for variable selection

Area of Science:

• Statistics
• Econometrics
• Machine Learning

Background:

• Traditional variable selection methods often yield a single model, making it difficult to assess uncertainty.
• Confidence intervals are standard for parameter estimation, but analogous concepts for model selection are less developed.

Purpose of the Study:

• Introduce Model Confidence Bounds (MCB) for variable selection in nested models.
• Provide a framework to quantify and visualize model selection uncertainty.
• Develop a graphical tool, the Model Uncertainty Curve (MUC), for assessing variability.

Main Methods:

• Develop the concept of MCB, defining upper and lower confidence bound models.
• Implement MCB using a fast bootstrap algorithm.
• Introduce the Model Uncertainty Curve (MUC) for visualization and comparison of model selection procedures.

Main Results:

• MCB identifies a set of nested models containing the true model with a given confidence level.
• The width and composition of MCB assess overall model selection uncertainty.
• The bootstrap algorithm provides correct asymptotic coverage under general conditions.

Conclusions:

• MCB offers a robust alternative to single-model selection, improving uncertainty assessment.
• The MUC provides valuable insights into model selection variability.
• The proposed methodology is validated by simulations and real-world data examples.