Related Experiment Video
Updated: May 12, 2026
06:28
E-Patient Counseling Trial (E-PACO): Computer Based Education versus Nurse Counseling for Patients to Prepare for Colonoscopy
Published on: August 1, 2019
Clinicians are right not to like Cohen's κ
Henrica C W de Vet1, Lidwine B Mokkink, Caroline B Terwee
1Department of Epidemiology and Biostatistics, EMGO Institute for Health and Care Research, VU University Medical Center, Amsterdam, Netherlands. hcw.devet@vumc.nl
BMJ (Clinical Research Ed.)
|April 16, 2013
Summary
Clinicians need absolute measures, not just reliability scores like Cohen's kappa, to understand observer variation. Percentage agreement, especially specific agreement, offers a clearer picture of interobserver and intraobserver agreement for categorical data.
Area of Science:
- Medical Statistics
- Clinical Research Methodology
Background:
- Clinicians require understanding observer variation (interobserver and intraobserver) for categorical outcomes.
- Cohen's kappa is frequently used but provides a relative measure of reliability, not absolute agreement.
Purpose of the Study:
- To evaluate the suitability of Cohen's kappa versus measures of agreement for assessing observer variation.
- To demonstrate the limitations of relative measures and advocate for absolute measures in clinical contexts.
Main Methods:
- Analysis of observer variation using a 2x2 table example.
- Comparison of Cohen's kappa (relative measure) with percentage agreement (absolute measure).
Main Results:
- Cohen's kappa can be misleading as it is a relative measure of reliability.
- Percentage agreement serves as an absolute measure of agreement, offering more clinically relevant information.
- Specific agreement (proportion of agreement for positive and negative ratings) is the most informative measure.
Conclusions:
- Clinicians should prioritize absolute measures of agreement over relative reliability measures like Cohen's kappa.
- Specific agreement in a 2x2 table provides the most useful data for understanding observer variation in clinical settings.
Related Concept Videos
01:17Cochran's Q Test
Cochran's Q Test is a nonparametric statistical test used to determine if there are potential differences in the outcomes of three or more related groups on a binary (yes/no) or dichotomous outcome. It is essentially an extension of the McNemar Test, which is limited to two related samples - Cochran's Q test can handle three or more related samples, making it more versatile in scenarios where subjects are measured under multiple conditions. The test statistic follows a Chi-Square distribution,...
01:20Kendall's Coefficient of Concordance
Kendall's Coefficient of Concordance (W), also known as Kendall's W, is a non-parametric statistical measure used to assess the agreement or concordance between multiple raters or judges when they rank a set of items. It is often used when you have ordinal data (ranks) and you want to see if there is consistency or consensus among the raters. It is widely applied in research areas such as psychology, medicine, and social sciences, where multiple judges are asked to rank or rate subjects or...
01:08Obedience
According to obedience research, we may harm others under the forceful pressures of an authority figure (Milgram, 1974). How about if the inappropriate orders were delivered with less force? The increasing interdependence between nurses and physicians compelled Hofling and his colleagues to explore nurses’ reactions to a potentially harmful medical request made by the perceived authority figure, the doctor (Hofling, Brotzman, Dalrymple, Graves, & Pierce, 1966). In this situation, obedience...
01:19The Mantel-Cox Log-Rank Test
The Mantel-Cox log-rank test is a widely used statistical method for comparing the survival distributions of two groups. It tests whether a statistically significant difference exists in survival times between the groups without assuming a specific distribution for the survival data, making it a non-parametric test. This flexibility makes the log-rank test particularly valuable in medical research and other fields where the timing of an event, such as death or disease recurrence, is of interest.
01:24Confidence Coefficient
The confidence coefficient is also known as the confidence level or degree of confidence. It is the percent expression for the probability, 1-α, that the confidence interval contains the true population parameter assuming that the confidence interval is obtained after sufficient unbiased sampling; for example, if the CL = 90%, then in 90 out of 100 samples the interval estimate will enclose the true population parameter. Here α is the area under the curve, distributed equally under both the...
02:20Cohesins
Cohesin protein complexes are a molecular glue that holds two sister chromatids together. They play an important role both in mitosis and meiosis. In mitosis, all cohesin complexes present on the chromosomes are removed before the start of the anaphase stage.
Cohesin complexes in Meiotic Division
Meiosis involves two distinct rounds of chromosomal segregation and cell divisions— Meiosis I followed by Meiosis II – producing four daughter cells. Meiosis I includes the separation of homologous...
Cohesin complexes in Meiotic Division
Meiosis involves two distinct rounds of chromosomal segregation and cell divisions— Meiosis I followed by Meiosis II – producing four daughter cells. Meiosis I includes the separation of homologous...