Factors associated with sensitive regression weights: A fungible parameter approach

Area of Science:

• Statistics
• Econometrics
• Machine Learning

Background:

• Parameter sensitivity challenges the reliability of scientific models.
• Estimating model parameters with certainty is crucial for accurate inferences.

Purpose of the Study:

• To introduce and utilize fungible weights for assessing parameter sensitivity.
• To quantify the impact of predictor correlations and variance inflation factors (VIF) on parameter weight stability.

Main Methods:

• Examining sets of interchangeable, slightly suboptimal linear regression weights.
• Calculating the range of fungible weights and correlating it with predictor characteristics (correlation, VIF).
• Developing an R function to compute fungible weight ranges from covariance matrices.

Main Results:

• In two-predictor models, predictor correlations almost entirely explain fungible weight ranges (R² = .990).
• Including VIF and interactions in two-predictor models perfectly explains weight ranges (R² = 1).
• In three-predictor models, correlations explain 83.9% of weight ranges, and VIF/interactions explain 91.0%.

Conclusions:

• High correlations among predictors allow for compensatory weight adjustments without significantly altering model predictions.
• Parameter sensitivity is influenced by inter-predictor relationships and VIF, affecting the trustworthiness of effect estimates.
• The study provides a method and tool (R function) for evaluating parameter sensitivity in regression analyses.