Estimation of required sample size for external validation of risk models for binary outcomes

Area of Science:

• Biostatistics
• Health Informatics
• Clinical Epidemiology

Background:

• External validation of health risk-prediction models is essential for clinical decision-making.
• Determining appropriate sample size is a critical aspect of designing validation studies.
• Key performance measures include discrimination (C-statistic) and calibration (slope, in the large).

Purpose of the Study:

• To investigate sample size requirements for validation studies with binary outcomes.
• To estimate measures of predictive performance with sufficient precision.
• To achieve adequate power for detecting differences from target values.

Main Methods:

• Developed simple estimators for sample size calculations under normality assumptions.
• Utilized simulation studies to evaluate estimator performance.
• Proposed non-normality-based estimators requiring numerical integration.

Main Results:

• Normality-based estimators perform well for common C-statistics and outcome prevalences, with slight performance degradation when assumptions are violated.
• Non-normality-based estimators also perform well under marginal normality.
• Sample size requirements vary based on prognostic strength, prevalence, performance measure, and study objective.

Conclusions:

• The proposed estimators provide a basis for sample size calculations in health model validation.
• Specific examples illustrate sample size needs for achieving desired precision in performance measures.
• The methods are applicable to survival outcomes with high censoring rates.