Using regression models for prediction: shrinkage and regression to the mean

Area of Science:

• Statistics
• Biostatistics
• Epidemiology

Background:

• Regression models are used for predicting future cases and risk assessment.
• Regression to the mean is a statistical phenomenon where future values tend to be closer to the mean.

Purpose of the Study:

• To discuss the implications of regression to the mean in predictive modeling.
• To study the extent of shrinkage in multiple and logistic regression models.
• To identify factors influencing shrinkage, such as sample size and number of covariates.

Main Methods:

• Analysis of fitted regression models, including multiple and logistic regression.
• Investigation of the relationship between shrinkage and goodness-of-fit statistics.
• Illustration of predictor shrinkage with practical examples.

Main Results:

• Shrinkage of predicted values towards the overall mean is a key characteristic of regression models.
• The degree of shrinkage is linked to goodness-of-fit measures.
• Shrinkage poses a significant challenge in small sample sizes or when using numerous covariates.

Conclusions:

• Regression to the mean can lead to over-optimistic predictions, particularly in small datasets or complex models.
• Understanding and quantifying shrinkage is crucial for accurate risk assessment and diagnostic tools.
• The study suggests a more general formulation to address shrinkage in regression analysis.