The cost of checking proportional hazards

Confidence intervals (CIs) and the reported predictive ability of statistical models may be misleading if one ignores uncertainty in the model selection procedure. When analyzing time-to-event data using Cox regression, one typically checks the proportional hazards (PH) assumption and subsequently alters the model to address any violations. Such an examination and correction constitute a model selection procedure, and, if not accounted for, could result in misleading CI. With the bootstrap, I study the impact of checking the PH assumption using (1) data to predict AIDS-free survival among HIV-infected patients initiating antiretroviral therapy and (2) simulated data. In the HIV study, due to non-PH, a Cox model was stratified on age quintiles. Interestingly, bootstrap CIs that ignored the PH check (always stratified on age quintiles) were wider than those which accounted for the PH check (on each bootstrap replication PH was tested and corrected through stratification only if violated). Simulations demonstrated that such a phenomenon is not an anomaly, although on average CIs widen when accounting for the PH check. In most simulation scenarios, coverage probabilities adjusting and not adjusting for the PH check were similar. However, when data were generated under a minor PH violation, the 95 per cent bootstrap CI ignoring the PH check had a coverage of 0.77 as opposed to 0.95 for CI accounting for the PH check. The impact of checking the PH assumption is greatest when the p-value of the test for PH is close to the test's chosen Type I error probability.