测量协议在几个评级者分类主体到一个或多个 (等级) 类别:一个泛化弗莱斯的卡帕Fleiss的卡帕
Filip Moons1,2, Ellen Vandervieren3
1Freudenthal Institute, Utrecht University, PO Box 85170, 3508 AD, Utrecht, the Netherlands. f.moons@uu.nl.
Behavior research methods
|September 15, 2025
概括
一个新的泛化Fleiss kappa统计允许多个类别的分配,以提高评级者之间的可靠性. 这种方法增强了对复杂数据的协议措施,如多个精神病诊断或编码方案.
科学领域:
- 统计 统计 统计 统计
- 心理测量 心理测量 心理测量
- 数据分析 数据分析
背景情况:
- 传统的评级者间协议措施,如科恩和弗莱斯的kappa限制评级者每科目单一类别的分配.
- 这种限制在精神病学或内容分析等领域是有问题的,因为受试者可以同时属于多个类别.
研究的目的:
- 提出Fleiss' kappa的概括版本,以适应每个主题的多个类别分配.
- 开发一个灵活的协议统计,可以纳入类别权重,层次结构,处理缺少的数据和不同数量的评级者.
主要方法:
- 开发了一个通用的卡帕统计,扩展弗莱斯的卡帕,用于多个名义类别分配.
- 导出了新的统计数据,证明其在单次分配条件下与Fleiss的kappa相当.
- 审查了多个类别分配的现有方法,并详细说明了新措施的假设和解释.
主要成果:
- 提出的一般化kappa统计有效地处理多个类别分配的情况.
- 该措施是灵活的,包括类别权重,层次结构,缺少的数据和可变的评级者数量.
- 当只分配单个类别时,可以证明与Fleiss' kappa的等价性.
结论:
- 一般化kappa统计提供了一个强大的解决方案,在多个类别的分配是必要的.
- 这项新措施提高了复杂的诊断和分类任务的可靠性评估.
- 提供R脚本和Excel表,用于研究中的实际应用,包括精神病诊断和评估方法.
相关概念视频
Kendall's Coefficient of Concordance
969
Kendall's Coefficient of Concordance (W), also known as Kendall's W, is a non-parametric statistical measure used to assess the agreement or concordance between multiple raters or judges when they rank a set of items. It is often used when you have ordinal data (ranks) and you want to see if there is consistency or consensus among the raters. It is widely applied in research areas such as psychology, medicine, and social sciences, where multiple judges are asked to rank or rate subjects...
969
Friedman Two-way Analysis of Variance by Ranks
480
Friedman's Two-Way Analysis of Variance by Ranks is a nonparametric test designed to identify differences across multiple test attempts when traditional assumptions of normality and equal variances do not apply. Unlike conventional ANOVA, which requires normally distributed data with equal variances, Friedman's test is ideal for ordinal or non-normally distributed data, making it particularly useful for analyzing dependent samples, such as matched subjects over time or repeated measures...
480
Kendall's Tau Test
1.1K
Kendall's tau test, also known as the Kendall rank coefficient test, is a nonparametric method for assessing association between two variables. This test is particularly useful for identifying significant correlations when the distributions of the sample and population are unknown. Developed in 1938 by the British statistician Sir Maurice George Kendall, the tau coefficient (denoted as τ) serves as a rank correlation coefficient, with values ranging from -1 to +1.
A τ value of +1 indicates...
A τ value of +1 indicates...
1.1K
Ratio Level of Measurement
20.6K
The way a set of data is measured is called its level of measurement. Correct statistical procedures depend on a researcher being familiar with levels of measurement. For analysis, data are classified into four levels of measurement—nominal, ordinal, interval, and ratio.
A set of data measured using the ratio scale takes care of the ratio problem and provides complete information. Ratio scale data are like interval scale data, except they have a zero point and ratios can be calculated....
A set of data measured using the ratio scale takes care of the ratio problem and provides complete information. Ratio scale data are like interval scale data, except they have a zero point and ratios can be calculated....
20.6K
Bonferroni Test
3.3K
The Bonferroni test is a statistical test named after Carlo Emilio Bonferroni, an Italian mathematician best known for Bonferroni inequalities. This statistical test is a type of multiple comparison test to determine which means are different than the rest. Bonferroni test can minimize the Type 1 error by reducing the significance level alpha, which otherwise increases with sample pairs.
The means of different samples are first paired in all possible combinations.
The null hypothesis of the...
The means of different samples are first paired in all possible combinations.
The null hypothesis of the...
3.3K
Ordinal Level of Measurement
32.0K
The way a set of data is measured is called its level of measurement. Correct statistical procedures depend on a researcher being familiar with levels of measurement. For analysis, data are classified into four levels of measurement—nominal, ordinal, interval, and ratio.
Data measured using an ordinal scale are similar to nominal scale data, but there is one major difference. The ordinal scale data can be ordered. An example of ordinal scale data is a list of the top five national parks...
Data measured using an ordinal scale are similar to nominal scale data, but there is one major difference. The ordinal scale data can be ordered. An example of ordinal scale data is a list of the top five national parks...
32.0K


