Variable selection for clinical prediction models in low-dimensional data - a simulation study comparing traditional...

Area of Science:

• Biostatistics
• Machine Learning in Healthcare
• Predictive Modeling

Background:

• Developing accurate clinical prediction models (CPMs) involves various variable selection techniques.
• Evaluating the performance of these methods in low-dimensional settings is essential for reliable predictions.

Purpose of the Study:

• To design a fair simulation study comparing traditional and machine learning variable selection methods for CPM development.
• To investigate the strengths and weaknesses of different approaches in predicting continuous outcomes.

Main Methods:

• A simulation study compared linear regression with stepwise selection (LMSS) against machine learning methods: elastic net, gradient boosting, and random forest.
• Datasets were generated with varying complexity, including linear, non-linear, and non-additive associations typical in biomedicine.

Main Results:

• Model performance improved with larger sample sizes and reduced data noise across all methods.
• LMSS demonstrated the best performance in selecting relevant predictors while excluding irrelevant variables in most scenarios.
• Gradient boosting and elastic net tended to include most variables, particularly with larger sample sizes.

Conclusions:

• Sufficient sample size is critical for reliable predictor identification and accurate, well-calibrated CPMs.
• LMSS is a robust method, and random forest with Boruta or Hapfelmeier approaches are viable alternatives for complex predictor-outcome associations.