Minimum sample size for external validation of a clinical prediction model with a continuous outcome

Area of Science:

• Biostatistics
• Clinical Epidemiology
• Health Informatics

Background:

• Clinical prediction models (CPMs) aid in patient counseling and decision-making by providing individualized outcome predictions.
• External validation is essential for assessing CPM performance on independent data, but often lacks sufficient sample size for reliable estimates.
• Small sample sizes in external validation studies lead to imprecise estimations of predictive performance.

Purpose of the Study:

• To propose a method for determining the minimum sample size required for the external validation of clinical prediction models with continuous outcomes.
• To ensure precise estimation of key performance metrics, including R-squared, calibration-in-the-large, calibration slope, and outcome variance.

Main Methods:

• Derivation of closed-form sample size solutions based on four criteria: R-squared, calibration-in-the-large, calibration slope, and outcome variance.
• The method requires users to specify anticipated performance values (e.g., R-squared) and outcome variance for the external dataset.
• The largest sample size calculated across all criteria is recommended as the minimum required sample size.

Main Results:

• The proposed method provides specific sample size calculations for ensuring precise estimates of CPM performance during external validation.
• The largest sample size needed to meet all four proposed criteria is identified as the minimum requirement.
• The approach can also assess the adequacy of existing datasets by estimating expected precision.

Conclusions:

• This study offers a robust framework for calculating the minimum sample size for external validation of clinical prediction models.
• Adhering to these sample size recommendations will enhance the reliability and precision of external validation studies.
• Accurate external validation is critical for the trustworthy application of clinical prediction models in practice.