A Covariance-Based Penalty Estimator for Model Assessment With Censored Data

Prediction model selection and assessment are primary objectives of many statistical analyses. Covariance-based penalty estimators provide analytic estimates of the optimism associated with naive training error estimates for multiple classes of prediction models and error assessment rules. While the majority of work on covariance-based penalties has focused on prediction for uncensored data, little attention has been given to time-to-event data. In this article, we consider estimating the optimism for survival prediction models assessed via the Brier score. We first analytically derive an expression of the optimism in a single group scenario with uncensored data based on a reformulation of the optimism. With the same reformulation, we propose an algorithm to estimate the optimism for Cox's proportional hazards regression under a general prediction setting involving covariates and right censoring. We verify the derived theory and demonstrate the applicability of the proposed algorithm via simulation studies. Finally, we illustrate the utility of our new covariance-based penalty estimator through an application predicting time to hemodialysis access failure among patients with end-stage renal disease using data from the United States Renal Data System.