Adaptive sample size determination for the development of clinical prediction models

Area of Science:

• Biostatistics
• Clinical Epidemiology
• Medical Informatics

Background:

• Developing robust clinical prediction models requires appropriate sample sizes.
• Traditional fixed sample size calculations may not be optimal for evolving datasets.
• Sequential monitoring of model performance can inform adaptive sample size determination.

Purpose of the Study:

• To propose and evaluate an adaptive sample size calculation method for clinical prediction models.
• To compare the adaptive method with fixed sample size rules and assess its performance in real-world datasets.
• To determine the sample size needed to achieve desired model performance criteria.

Main Methods:

• An adaptive sample size method was developed, monitoring model performance sequentially as new data arrived.
• The method was illustrated using datasets for ovarian cancer and obstructive coronary artery disease (CAD).
• Model performance was assessed using calibration slope and optimism in the c-statistic (AUC) with bootstrapping for internal validation.

Main Results:

• Adequate calibration and limited optimism were achieved with a median of 450 patients for ovarian cancer and 850 for CAD.
• Stricter criteria required higher sample sizes, with medians of 500 and 1500 patients, respectively.
• The adaptive method yielded sample sizes higher than the 10 events per parameter (EPP) rule but comparable to other advanced methods.

Conclusions:

• Adaptive sample size determination is a valuable supplement to fixed calculations.
• It allows dynamic tailoring of sample size to specific prediction modeling contexts.
• This approach enhances the reliability and efficiency of clinical prediction model development.