Confidence intervals for the Cox model test error from cross-validation.

Area of Science:

• Statistical learning
• Machine learning model evaluation
• Survival analysis

Background:

• Cross-validation (CV) is a standard method for estimating model test error.
• Standard CV may produce confidence intervals with inadequate coverage due to correlated error estimates.
• This underestimation arises because data samples are used for both training and testing.

Purpose of the Study:

• To address the coverage issues in standard cross-validation for model evaluation.
• To generalize the nested cross-validation approach for improved variance estimation.
• To explore test error estimation for the Cox proportional hazards model using nested CV.

Main Methods:

• Implementing nested cross-validation (nested CV) to estimate prediction error.
• Calculating mean squared error of prediction error to account for correlations.
• Applying nested CV to the Cox proportional hazards model framework.

Main Results:

• Nested CV demonstrates superior confidence interval coverage compared to standard CV.
• The proposed method mitigates the underestimation of variance in test error estimates.
• The study explores various test error metrics within the Cox model context.

Conclusions:

• Nested CV offers a more reliable method for assessing model performance and uncertainty.
• This technique is particularly valuable for the Cox proportional hazards model in survival analysis.
• Accounting for correlations in CV estimates is crucial for accurate model evaluation.