Training replicable predictors in multiple studies.

Area of Science:

• Machine Learning
• Biostatistics
• Computational Biology

Background:

• Replicability of prediction model performance across studies is a significant challenge in scientific research.
• Current practices often rely on single-study training, potentially limiting generalizability and robustness.

Purpose of the Study:

• To propose and evaluate a general approach for investigating prediction performance replicability using ensembles of models.
• To quantify the impact of single-study training on replicability issues.
• To explore the development of robust ensemble learners that incorporate replicability.

Main Methods:

• Training prediction models on diverse datasets from multiple studies.
• Utilizing ensemble learning techniques to combine predictions from different models.
• Developing criteria for combining multi-study ensembles to enhance robustness.

Main Results:

• Single-study training partially explains observed difficulties in replicability.
• Ensembles of predictors trained on multiple studies demonstrate improved replicability.
• Proposed criteria enable the design of upfront ensemble learners for enhanced replicability.

Conclusions:

• Ensemble methods offer a promising strategy to address the challenge of prediction performance replicability.
• Training on multiple studies and employing specific combination criteria can lead to more robust and generalizable prediction models.
• This approach can improve the reliability of predictive models in various contexts and populations.