Performance of binary prediction models in high-correlation low-dimensional settings: a comparison of methods

Area of Science:

• Medical Informatics
• Biostatistics

Background:

• Clinical prediction models are crucial in medicine.
• High collinearity among predictors can lead to spurious associations and reduce model face-validity.
• Excluding collinear predictors without a priori justification can be arbitrary.

Purpose of the Study:

• To compare methods for addressing collinearity in clinical prediction models.
• To evaluate the impact of collinearity on model performance and predictor selection stability.

Main Methods:

• Compared shrinkage, dimensionality reduction, and constrained optimization techniques.
• Utilized simulations to assess method effectiveness under varying collinearity levels.

Main Results:

• Collinearity did not affect predictive outcomes (AUC, R², Intercept, Slope).
• High collinearity negatively impacted predictor selection stability across all methods, especially those with strong selection (e.g., Lasso).
• Ridge, PCLR, LAELR, and Dropout demonstrated the most stable predictor sets under collinearity.

Conclusions:

• Advise against data-driven predictor selection in high collinearity settings due to selection instability.
• Warn that selected predictors may appear more strongly associated with outcomes than excluded ones, creating a false impression.