A jackknife approach to estimate the prediction uncertainty from binary classifiers under right-censoring

Area of Science:

• Statistics
• Machine Learning
• Biostatistics

Background:

• Clinical prediction models estimate patient risk for outcomes over time.
• Machine learning enhances prediction accuracy, but assessing uncertainty with right-censoring (using inverse-probability-of-censoring-weighting) is challenging.

Purpose of the Study:

• To propose an adjusted infinitesimal jackknife estimator for prediction standard errors that accounts for inverse-probability-of-censoring-weighting.
• To provide a broadly applicable nonparametric method, especially for machine learning classifiers.

Main Methods:

• Developed an adjusted infinitesimal jackknife estimator incorporating inverse-probability-of-censoring-weighting.
• Evaluated performance via simulation studies using parametric and machine learning models.
• Applied the method to predict post-transplant survival probabilities using registry data.

Main Results:

• The proposed adjustment yields unbiased standard error estimates in a tractable example.
• The adjusted estimator effectively quantifies prediction uncertainty for inverse-probability-of-censoring-weighting classifiers.
• Prediction uncertainty is higher with binary classifiers on dichotomized data compared to survival models.

Conclusions:

• The proposed infinitesimal jackknife adjustment is a valuable tool for assessing prediction uncertainty in time-to-event analyses with inverse-probability-of-censoring-weighting.
• This method enhances the reliability of risk predictions from machine learning models in survival data.
• Findings highlight the importance of appropriate uncertainty quantification for clinical decision-making.