The kappa statistic in reliability studies: use, interpretation, and sample size requirements
Related Experiment Video
Updated: Aug 19, 2026
Doppler Ultrasound-Based Leg Blood Flow Assessment During Single-Leg Knee-Extensor Exercise in an Uncontrolled Setting
Published on: December 15, 2023
1Primary Care Sciences Research Centre, Keele University, Keele, Staffordshire ST5 5BG, United Kingdom. j.sim@keele.ac.uk
Purpose:
This article examines and illustrates the use and interpretation of the kappa statistic in musculoskeletal research.
Summary Of Key Points:
The reliability of clinicians' ratings is an important consideration in areas such as diagnosis and the interpretation of examination findings. Often, these ratings lie on a nominal or an ordinal scale. For such data, the kappa coefficient is an appropriate measure of reliability. Kappa is defined, in both weighted and unweighted forms, and its use is illustrated with examples from musculoskeletal research. Factors that can influence the magnitude of kappa (prevalence, bias, and non-independent ratings) are discussed, and ways of evaluating the magnitude of an obtained kappa are considered. The issue of statistical testing of kappa is considered, including the use of confidence intervals, and appropriate sample sizes for reliability studies using kappa are tabulated.
Conclusions:
The article concludes with recommendations for the use and interpretation of kappa.
Related Concept Videos
01:16Kendall's Tau Test
A τ value of +1 indicates that...
01:13Accuracy and Errors in Hypothesis Testing
In hypothesis testing, the probability of making a Type I error, denoted as α, is commonly set at 0.05. This significance level indicates a 5% chance...
01:20Kendall's Coefficient of Concordance
01:16Critical Region, Critical Values and Significance Level
In hypothesis testing, a sample statistic is converted to a test statistic using z, t, or chi-square distribution. A critical region is an area under the curve in probability distributions demarcated by the critical value. When the test statistic falls in this region, it suggests that the null hypothesis must be rejected. As this region contains all those values of the test...
01:16Goodness-of-Fit Test
01:19Interpretation of Confidence Intervals
Confidence intervals have confidence coefficients that are crucial for their interpretation. The most common confidence coefficients are 0.90, 0.95, and 0.99, which can be written as percentages–90%, 95%, and 99%, respectively.
Suppose a person calculates a confidence interval with a confidence coefficient of 0.95. In that case, they can...