Logistic regression models for patient-level prediction based on massive observational data: Do we need all data?

Area of Science:

• Health Informatics
• Biostatistics
• Machine Learning in Healthcare

Background:

• Developing accurate predictive models in healthcare is crucial for clinical decision-making.
• Optimizing sample size is essential to balance model performance, complexity, and computational cost.

Purpose of the Study:

• To empirically determine adequate sample sizes for developing predictive models.
• To guide sample size considerations by balancing model performance and complexity.

Main Methods:

• Learning curves were generated for 17,248 prediction models across 81 prediction problems in depression and hypertension cohorts.
• Adequate sample size was defined as the point where model performance reached maximum performance minus a small threshold.

Main Results:

• A median reduction in observations ranged from 9.5% to 78.5% for thresholds of 0.001 to 0.02.
• Median reduction in model predictors ranged from 8.6% to 68.3% for the same thresholds.
• Substantial reductions in sample size and model complexity are achievable.

Conclusions:

• A fraction of available data is often sufficient for developing high-performing predictive models.
• Reduced sample sizes lead to substantially decreased model complexity and computational requirements.
• Learning curves can enable significant model complexity reduction, varying by outcome.