Related Experiment Video
Updated: Jul 13, 2026
08:12
A Psychophysics Paradigm for the Collection and Analysis of Similarity Judgments
Published on: March 1, 2022
Modelling patterns of agreement for nominal scales
1Biostatistics Group, School of Medicine, University of Manchester, Stopford Building, Oxford Road, Manchester M13 9PL, UK. chris.roberts@manchester.ac.uk
Statistics in Medicine
|July 20, 2007
Summary
This study introduces novel kappa-type coefficients to analyze agreement patterns in categorical scales, offering deeper insights beyond a single summary measure. These methods are crucial for reliability assessment and genetic twin studies.
Area of Science:
- Statistics
- Biostatistics
- Psychometrics
Background:
- Reliability of categorical scales is typically assessed by repeat rating agreement.
- Agreement measurement is vital in genetic twin studies using categorical scales.
- The kappa coefficient is a common analysis method for both applications.
Purpose of the Study:
- To model heterogeneity in agreement patterns for categorical scales with more than two categories.
- To develop kappa-type coefficients for analyzing varying agreement across category pairs.
- To provide methods for estimation, confidence intervals, inference, sample size, and power calculations.
Main Methods:
- Utilized kappa-type coefficients to model heterogeneity in agreement patterns.
- Introduced constraints to simplify the heterogeneous model.
- Derived formulae for sample size and power using the non-central chi-squared distribution.
Main Results:
- Developed procedures for estimation, confidence intervals, and inference for two independent samples.
- Simulation studies confirmed the empirical test size and power of the proposed methods.
- Illustrated methods with examples using nominal scales featuring three categories.
Conclusions:
- Kappa-type coefficients effectively model heterogeneity in agreement patterns for categorical scales.
- The proposed methods offer enhanced insights compared to single summary kappa coefficients.
- The study provides a comprehensive framework for analyzing complex agreement structures.
Related Concept Videos
00:56Nominal Level of Measurement
The way a set of data is measured is called its level of measurement. Correct statistical procedures depend on a researcher being familiar with levels of measurement. Not every statistical operation can be used with every set of data. For analysis, data are classified into four levels of measurement—nominal, ordinal, interval, and ratio.
The data that cannot be measured but can be grouped into categories fall under the nominal level of measurement. Data that is measured using a nominal scale is...
The data that cannot be measured but can be grouped into categories fall under the nominal level of measurement. Data that is measured using a nominal scale is...
00:55Ordinal Level of Measurement
The way a set of data is measured is called its level of measurement. Correct statistical procedures depend on a researcher being familiar with levels of measurement. For analysis, data are classified into four levels of measurement—nominal, ordinal, interval, and ratio.
Data measured using an ordinal scale are similar to nominal scale data, but there is one major difference. The ordinal scale data can be ordered. An example of ordinal scale data is a list of the top five national parks in the...
Data measured using an ordinal scale are similar to nominal scale data, but there is one major difference. The ordinal scale data can be ordered. An example of ordinal scale data is a list of the top five national parks in the...
01:12Sign Test for Nominal Data
The sign test is a nonparametric method used to evaluate hypotheses about the median of a single sample or to compare the medians of two related samples. The sign test is particularly useful when dealing with nominal data, which includes distinct categories without an inherent order, such as names, labels, and preferences. Nominal data restricts statistical analysis to evaluating population proportions rather than mean or median values that require continuous data.
For example, consider a...
For example, consider a...
01:07What is a Mode?
The mode is one of the commonly used measures of a central tendency. It is defined as the most frequent value in a data set.
There can be more than one mode in a data set if multiple values have the same highest frequency. For instance, suppose that the Statistics exam scores of 20 students are: 50; 53; 59; 59; 63; 63; 72; 72; 72; 72; 72; 76; 78; 81; 83; 84; 84; 84; 90; 93. Here, the mode is 72, as it occurs most frequently, five times.
A data set with two modes is called bimodal. For example,...
There can be more than one mode in a data set if multiple values have the same highest frequency. For instance, suppose that the Statistics exam scores of 20 students are: 50; 53; 59; 59; 63; 63; 72; 72; 72; 72; 72; 76; 78; 81; 83; 84; 84; 84; 90; 93. Here, the mode is 72, as it occurs most frequently, five times.
A data set with two modes is called bimodal. For example,...
01:02Ranks
Unlike parametric methods, nonparametric statistics are ideal for nominal and ordinal data, requiring fewer assumptions about the population's nature or distribution. This makes nonparametric methods easier to apply and interpret, as they do not depend on parameters like mean or standard deviation. One common approach in nonparametric analysis is to sort data according to a specific criterion. For instance, we might arrange weather data from hottest to coldest days in a month or rank cities...
01:21Friedman Two-way Analysis of Variance by Ranks
Friedman's Two-Way Analysis of Variance by Ranks is a nonparametric test designed to identify differences across multiple test attempts when traditional assumptions of normality and equal variances do not apply. Unlike conventional ANOVA, which requires normally distributed data with equal variances, Friedman's test is ideal for ordinal or non-normally distributed data, making it particularly useful for analyzing dependent samples, such as matched subjects over time or repeated measures from...