The relationship between the C-statistic and the accuracy of program-specific evaluations

Area of Science:

• Transplantation outcomes research
• Health services research
• Biostatistics

Background:

• The C-statistic is commonly used to evaluate risk-adjustment models in program evaluations.
• Its reliance on individual transplant risk variability may limit its appropriateness for assessing overall program accuracy.

Purpose of the Study:

• To investigate the association between the C-statistic and program evaluation accuracy metrics.
• To determine the reliability of the C-statistic for categorizing programs and identifying those needing regulatory review.

Main Methods:

• A simulation study using deceased donor kidney-alone transplant data (adult recipients, January 2018 reports).
• Generated various C-statistics by altering individual transplant risk variability.
• Assessed correlations between C-statistics, program evaluation metrics, and the number of expected events.

Main Results:

• The C-statistic showed no association with any metric of program evaluation accuracy.
• The number of expected events was the most significant factor influencing accuracy.
• Correlation between true hazard ratios and assigned tiers varied significantly based on expected events (<3 vs. >10).

Conclusions:

• The C-statistic offers no reliable information regarding the accuracy of transplant program evaluations.
• The number of expected events is a more critical determinant for assessing program performance and guiding regulatory decisions.