The concordance index C and the Mann-Whitney parameter Pr(X>Y) with randomly censored data

James A Koziol1, Zhenyu Jia

  • 1Department of Molecular and Experimental Medicine, The Scripps Research Institute, La Jolla, CA 92037, USA.

Insights

Harrell's c-index estimates survival distribution separation without censoring. However, with random censoring, it no longer estimates the Mann-Whitney parameter, unlike Efron's recommended estimator.

Area of Science:

  • Statistics
  • Biostatistics
  • Survival Analysis

Background:

  • Harrell's c-index (concordance C) is a common metric for assessing separation between two survival distributions.
  • In scenarios without data censoring, the c-index accurately estimates the Mann-Whitney parameter, Pr(X>Y).
  • This parameter has broad applications across various statistical fields.

Purpose of the Study:

  • To clarify the behavior of Harrell's c-index in the presence of randomly censored data.
  • To highlight the distinction between the c-index and the Mann-Whitney parameter under random censorship.
  • To introduce Efron's maximum likelihood estimator as a recommended alternative for the Mann-Whitney parameter in censored data settings.

Main Methods:

  • Comparative analysis of Harrell's c-index and Efron's maximum likelihood estimator.
  • Theoretical examination of survival distribution separation metrics.
  • Evaluation of parameter estimation under random censorship.

Main Results:

  • Harrell's c-index does not estimate the Mann-Whitney parameter Pr(X>Y) when data is randomly censored.
  • Under random censorship, the c-index estimates a different parameter influenced by censoring distributions.
  • Efron's maximum likelihood estimator remains a valid method for estimating the Mann-Whitney parameter with censored data.

Conclusions:

  • Harrell's c-index interpretation requires careful consideration of data censoring.
  • Efron's estimator is preferred for accurate Mann-Whitney parameter estimation in the presence of random censorship.
  • Understanding these distinctions is crucial for reliable survival data analysis.

Related Concept Videos

Censoring Survival Data01:09

Censoring Survival Data

Survival analysis is a statistical method used to analyze time-to-event data, often employed in fields such as medicine, engineering, and social sciences. One of the key challenges in survival analysis is dealing with incomplete data, a phenomenon known as "censoring." Censoring occurs when the event of interest (such as death, relapse, or system failure) has not occurred for some individuals by the end of the study period or is otherwise unobservable, and it might have many different reasons...
Kendall's Coefficient of Concordance01:20

Kendall's Coefficient of Concordance

Kendall's Coefficient of Concordance (W), also known as Kendall's W, is a non-parametric statistical measure used to assess the agreement or concordance between multiple raters or judges when they rank a set of items. It is often used when you have ordinal data (ranks) and you want to see if there is consistency or consensus among the raters. It is widely applied in research areas such as psychology, medicine, and social sciences, where multiple judges are asked to rank or rate subjects or...
Wilcoxon Rank-Sum Test01:21

Wilcoxon Rank-Sum Test

The Wilcoxon rank-sum test, also known as the Mann-Whitney U test, is a nonparametric test used to determine if there is a significant difference between the distributions of two independent samples. This test is designed specifically for two independent populations and has the following key requirements:
Friedman Two-way Analysis of Variance by Ranks01:21

Friedman Two-way Analysis of Variance by Ranks

Friedman's Two-Way Analysis of Variance by Ranks is a nonparametric test designed to identify differences across multiple test attempts when traditional assumptions of normality and equal variances do not apply. Unlike conventional ANOVA, which requires normally distributed data with equal variances, Friedman's test is ideal for ordinal or non-normally distributed data, making it particularly useful for analyzing dependent samples, such as matched subjects over time or repeated measures from...
Wilcoxon Signed-Ranks Test for Median of Single Population01:14

Wilcoxon Signed-Ranks Test for Median of Single Population

The Wilcoxon signed-rank test for the median of a single population is a nonparametric test used to evaluate whether the median of a population differs from a specified value. Unlike parametric tests, it does not require data to follow a normal distribution, making it suitable for non-normal or small samples. The test begins by calculating the difference (d) between each observation and the hypothesized median. The absolute values of these differences are ranked in ascending order, with ties...
Calculating and Interpreting the Linear Correlation Coefficient01:11

Calculating and Interpreting the Linear Correlation Coefficient

The correlation coefficient, r, developed by Karl Pearson in the early 1900s, is numerical and provides a measure of strength and direction of the linear association between the independent variable, x, and the dependent variable, y. Hence, it is also known as the Pearson product-moment correlation coefficient. It can be calculated using the following equation: