Related Experiment Video
Updated: Jul 17, 2026

A Psychophysics Paradigm for the Collection and Analysis of Similarity Judgments
Published on: March 1, 2022
[Comparative analysis of three approaches for rater agreement]
Manuel Ato1, Ana Benavente, Juan J López
1Univesidad de Murcia, Spain. matogar@um.es
Abstract:
This work is a comparative analysis of three general approaches for rater agreement measurement which produce three different statistical coefficients: descriptive (sigma, pi y kappa coefficients), based on log-linear models and on mixture models. Although loglinear and mixture approaches assume a different concept of chance correction, their flexibility allow using its parameters to analyze agreement and disagreement patterns. We show that all descriptive measures can also be obtained by introducing constraints to a basic loglinear (or mixture) quasi-independence model. Due to the different measurement scale used, both loglinear and mixture approaches have a similar interpretation when diagonal parameters are positive but a different interpretation otherwise.
Related Concept Videos
Friedman Two-way Analysis of Variance by Ranks
Kendall's Coefficient of Concordance
One-Way ANOVA
Group Design
One-Way ANOVA: Equal Sample Sizes
Different sample means can result in different values for the variance estimate: variance between samples. This is because the variance between samples is calculated as the product of the sample size and the variance between the...
Ratio Level of Measurement
A set of data measured using the ratio scale takes care of the ratio problem and provides complete information. Ratio scale data are like interval scale data, except they have a zero point and ratios can be calculated. For...