Related Experiment Videos
Automated variable selection methods for logistic regression produced unstable models for predicting acute myocardial
1Institute for Clinical Evaluative Sciences, G1 06, 2075 Bayview Avenue, Toronto, Ontario M4N 3M5, Canada. peter.austin@ices.on.ca
Journal of Clinical Epidemiology
|November 30, 2004
Summary
Automated variable selection methods for predicting mortality after acute myocardial infarction (AMI) produce unstable models. These methods are not reproducible, as variable selection is highly sensitive to random data fluctuations.
Area of Science:
- Biostatistics
- Epidemiology
- Medical Informatics
Background:
- Automated variable selection is common in developing predictive models.
- Reproducibility of these models is crucial for clinical application.
Purpose of the Study:
- To assess the reproducibility of logistic regression models developed using automated variable selection.
- To determine the stability of predictor variables identified by these methods.
Main Methods:
- Utilized 1,000 bootstrap samples from a dataset of 4,911 acute myocardial infarction (AMI) patients.
- Applied backward elimination, forward selection, and stepwise selection for logistic regression models.
- Compared model agreement across methods and bootstrap samples for predicting 30-day mortality.
Main Results:
- Backward elimination generated 940 unique models across bootstrap samples.
- Forward and stepwise selection yielded similar variability in model identification.
- Only three variables consistently predicted mortality; over half were selected in less than 50% of samples.
Conclusions:
- Automated variable selection methods yield unstable and non-reproducible logistic regression models.
- Selected independent predictors are highly sensitive to random variations within the data.
- Clinical use of models derived from automated selection requires caution due to poor reproducibility.