Selection of variables for multivariable models: Opportunities and limitations in quantifying model stability by...

Area of Science:

• Statistics
• Biostatistics
• Epidemiology

Background:

• Statistical models are essential for describing outcome-covariate associations.
• Stepwise variable selection is common but can introduce instability and bias.
• Model stability is often overlooked despite the impact of variable selection.

Purpose of the Study:

• To evaluate the consistency and accuracy of resampling-based measures for statistical model stability.
• To compare subsampling and bootstrapping techniques for assessing model stability.
• To investigate the optimal choice of resampling method for different stability metrics.

Main Methods:

• Simulation study comparing subsampling and bootstrapping.
• Assessment of linear, logistic, and Cox models using backward elimination.
• Evaluation of variable inclusion frequencies (VIFs), model selection frequencies, relative conditional bias (RCB), and root mean squared difference ratio (RMSDR).

Main Results:

• Subsampling consistently estimates VIFs and model selection frequencies.
• Bootstrapping offers better bias and precision for estimating RCB and RMSDR.
• Accurate RMSDR estimation requires covariate independence, which is rare in practice.

Conclusions:

• Addressing model stability after variable selection is critical for reliable statistical modeling.
• The choice of resampling technique impacts the accuracy of stability estimates.
• Subsampling and bootstrapping have complementary strengths for assessing different aspects of model stability.