Embracing cohort heterogeneity in clinical machine learning development: a step toward generalizable models.

Area of Science:

• Biostatistics
• Machine Learning
• Epidemiology

Background:

• Prediction models are crucial in various scientific fields.
• Current guidelines often focus on single-cohort data for model development.
• The generalizability of models trained on limited data is a persistent challenge.

Purpose of the Study:

• To illustrate the benefits of averaging prediction models over multiple cohorts.
• To demonstrate improved performance of multi-cohort models in novel settings.
• To highlight the need for updated prediction model development guidelines.

Main Methods:

• Comparative analysis of prediction models.
• Training models on single-cohort versus multi-cohort data.
• Evaluating model performance on unseen data.

Main Results:

• Models trained on averaged multi-cohort data significantly outperform single-cohort models.
• Increased data diversity from multiple cohorts enhances predictive accuracy.
• The benefit of averaging is evident even with equivalent total training data size.

Conclusions:

• Averaging prediction models across cohorts is a simple yet powerful strategy.
• Multi-cohort model averaging enhances generalizability and robustness.
• Existing guidelines for prediction model development should incorporate multi-cohort approaches.