On the impact of model selection on predictor identification and parameter inference

Area of Science:

• Statistics
• Biostatistics
• Machine Learning

Background:

• Penalized regression methods are crucial for identifying outcome-associated predictors in high-dimensional data.
• Assessing predictor selection's impact on parameter inference is vital for practical applications with limited sample sizes.

Purpose of the Study:

• To evaluate penalized regression methods for predictor identification in linear and logistic models.
• To compare direct penalized estimates (Algorithm 1) versus refitted estimates (Algorithm 2) for parameter inference.

Main Methods:

• Assessed penalized linear regression, elastic net, SCAD, LASSO, partial least squares, and relaxo.
• Evaluated logistic regression variants including LASSO, SCAD, and adaptive logistic regression.
• Compared Algorithm 1 (direct penalized estimates) and Algorithm 2 (refitted selected predictors).

Main Results:

• Relaxo demonstrated superior performance with low false positive and false negative rates in linear models.
• LASSO and penalized logistic regression showed high false positive rates for logistic models.
• Algorithm 2 provided better confidence interval coverage than Algorithm 1, especially for large sample and effect sizes, though Algorithm 1 maintained 100% coverage for null effects.

Conclusions:

• Relaxo is a promising method for predictor selection in linear models.
• Algorithm 2 is preferred for parameter estimation but requires substantial sample and effect sizes for reliable statistical inference.
• Method performance varied significantly between linear and logistic models, highlighting the need for careful method selection.