Concordance Indices for Risk Scores With Policy Evaluations

To demonstrate the differences between concordance index (C-index) methodologies and clarify the appropriate usage for risk score evaluations in health services applications.

We performed a methodological comparison of C-index metrics and illustrated the consequences of these differences through a study of liver failure patients.

We analyzed secondary adult liver transplant registry data from the Organ Procurement and Transplantation Network (OPTN), including all waitlist registrations from 2002 to 2022.

The recommended concordance metric based on Gerds' weighting was higher for the original model for end-stage liver disease (MELD) than Harrell's C-Index, Uno's C-Index, and naïve binary outcome metrics (0.864 [95% confidence interval (CI): 0.840, 0.888] versus 0.854 [95% CI: 0.844, 0.864], 0.832 [95% CI: 0.819, 0.844], and 0.727 [95% CI: 0.715, 0.740]), and it did not increase after the latest MELD formula update (0.874 [95% CI: 0.859, 0.889] to 0.869 [95% CI: 0.853, 0.885]).

The concordance indices that are often used in health services applications have important deficiencies under policy-related dependent censoring, and researchers must apply appropriate weighting schemes to avoid bias. The findings uncover new interpretations of past evaluation results that have shaped national liver transplant policies.