Minimum sample size for external validation of a clinical prediction model with a binary outcome.

Area of Science:

• Biostatistics
• Clinical Epidemiology
• Health Informatics

Background:

• External validation is essential for assessing prediction model generalizability.
• Current external validation studies often lack sufficient sample sizes, leading to unreliable performance estimates.
• Precise estimation of model performance is critical for clinical decision-making.

Purpose of the Study:

• To propose a method for determining the minimum sample size required for external validation studies of prediction models for binary outcomes.
• To enable precise estimation of calibration, discrimination, and clinical utility.
• To provide guidance on assessing the adequacy of existing datasets for external validation.

Main Methods:

• Development of closed-form and iterative solutions for sample size calculation.
• Inclusion of key parameters: target standard errors, event proportion, model calibration, and risk thresholds.
• Application of the method to a case study of mechanical heart valve failure prediction.

Main Results:

• Calculations indicate a need for at least 9835 participants (177 events) for precise calibration and discrimination estimates, often limited by the calibration slope.
• A sample size of 6443 participants (116 events) is required for precise net benefit estimation at an 8% risk threshold.
• The proposed methods can inform sample size adequacy for already collected datasets.

Conclusions:

• The proposed sample size calculation methods enhance the reliability of external validation studies for prediction models.
• Accurate sample size determination ensures precise performance estimates, supporting confident clinical application of prediction models.
• Availability of software code facilitates the implementation of these sample size calculations.