Related Experiment Videos
Kappa coefficients in medical research
Helena Chmura Kraemer1, Vyjeyanthi S Periyakoil, Art Noda
1Department of Psychiatry and Behavioral Sciences, Stanford University School of Medicine, Stanford, California, U.S.A. hck@leland.stanford.edu
Abstract:
Kappa coefficients are measures of correlation between categorical variables often used as reliability or validity coefficients. We recapitulate development and definitions of the K (categories) by M (ratings) kappas (K x M), discuss what they are well- or ill-designed to do, and summarize where kappas now stand with regard to their application in medical research. The 2 x M(M>/=2) intraclass kappa seems the ideal measure of binary reliability; a 2 x 2 weighted kappa is an excellent choice, though not a unique one, as a validity measure. For both the intraclass and weighted kappas, we address continuing problems with kappas. There are serious problems with using the K x M intraclass (K>2) or the various K x M weighted kappas for K>2 or M>2 in any context, either because they convey incomplete and possibly misleading information, or because other approaches are preferable to their use. We illustrate the use of the recommended kappas with applications in medical research.
Insights
Kappa coefficients measure categorical variable correlation for reliability and validity. While 2xM intraclass kappa is ideal for binary reliability and 2x2 weighted kappa for validity, KxM kappas (K>2 or M>2) present significant issues in medical research.
Area of Science:
- Statistics
- Medical Research Methodology
Background:
- Kappa coefficients are widely used to assess inter-rater reliability and agreement between categorical variables.
- Their application in medical research is common for evaluating diagnostic tests and clinical assessments.
- Understanding the nuances of different kappa types (KxM) is crucial for accurate interpretation.
Purpose of the Study:
- To review the development and definitions of KxM kappa coefficients.
- To delineate the appropriate and inappropriate applications of various kappa measures.
- To provide guidance on the optimal use of kappa statistics in medical research.
Main Methods:
- Recapitulation of kappa coefficient development and definitions.
- Discussion of the design limitations and strengths of KxM kappas.
- Illustrative applications of recommended kappa measures in medical research.
Main Results:
- The 2xM intraclass kappa is identified as the optimal measure for binary reliability.
- A 2x2 weighted kappa is recommended as a strong, though not exclusive, choice for validity assessment.
- Significant problems exist with KxM kappas when K>2 or M>2, potentially yielding incomplete or misleading information.
Conclusions:
- Specific kappa coefficients are well-suited for binary reliability (2xM intraclass) and validity (2x2 weighted).
- KxM kappa coefficients with K>2 or M>2 are often problematic and should be used with caution or avoided.
- Alternative approaches may be preferable when dealing with more complex categorical agreement scenarios in medical research.