Exact and resampling probability values for weighted kappa
Related Experiment Video
Updated: Aug 17, 2026

A Psychophysics Paradigm for the Collection and Analysis of Similarity Judgments
Published on: March 1, 2022
Kenneth J Berry1, Janis E Johnston, Paul W Mielke
1Department of Sociology, Colorado State University, Fort Collins, CO 80523-1784, USA. berry@lamar.colostate.edu
Abstract:
Permutation procedures to compute exact and resampling probability values for weighted kappa are described. Comparisons with asymptotic probability values demonstrate that exact permutation procedures are advantageous for sparse data sets, whereas resampling permutation procedures are appropriate for both sparse and nonsparse data sets.
Related Concept Videos
Weighted Mean
For example, consider the number of goals scored in the matches of a tournament. While computing the average number of goals scored in the tournament, it may be more important to...
Kendall's Coefficient of Concordance
Expected Frequencies in Goodness-of-Fit Tests
Chebyshev's Theorem to Interpret Standard Deviation
Accuracy and Errors in Hypothesis Testing
In hypothesis testing, the probability of making a Type I error, denoted as α, is commonly set at 0.05. This significance level indicates a 5% chance...
Testing a Claim about Population Proportion
There are two methods of testing a claim about a population proportion: (1) Using the sample proportion from the data where a binomial distribution is approximated to the normal distribution and (2) Using the binomial probabilities calculated from the data.
The first method uses normal distribution as an approximation to the binomial distribution. The requirements are as follows: sample size is large...
