Related Experiment Videos
Resampling and cross-validation techniques: a tool to reduce bias caused by model building?
M Schumacher1, N Holländer, W Sauerbrei
1Institute of Medical Biometry and Informatics, University of Freiburg, Germany.
Statistics in Medicine
|March 4, 1998
Summary
Model building in medical studies can overestimate predictive accuracy. This study evaluates cross-validation and bootstrap resampling to reduce this bias in regression models, using breast cancer data and simulations.
Area of Science:
- Biostatistics
- Medical Informatics
- Computational Biology
Background:
- Over-optimism in predictive model performance is a common issue in medical research.
- The 'final' regression model's predictive ability is often overestimated due to model-building processes.
Purpose of the Study:
- To illustrate the over-optimism phenomenon in a simple cutpoint model.
- To assess bias reduction techniques like cross-validation and bootstrap resampling.
- To compare these intensive methods against ad hoc and heuristic approaches.
Main Methods:
- Illustration using a breast cancer study dataset.
- Application of cross-validation and bootstrap resampling.
- Comparison with ad hoc and heuristic bias reduction methods.
- Conducting a simulation study to evaluate method quality.
Main Results:
- Demonstration of over-optimism in regression model predictive ability.
- Quantification of bias reduction achieved by cross-validation and bootstrap resampling.
- Comparative performance analysis of different bias reduction strategies.
Conclusions:
- Cross-validation and bootstrap resampling are effective in mitigating over-optimism in medical predictive models.
- These computer-intensive methods offer improved reliability over simpler approaches.
- The findings are validated through both real-world data and simulation studies.