作为研究工具的比较判断:应用和可靠性的元分析
George Kinnear1, Ian Jones2, Ben Davies3
1School of Mathematics and Maxwell Institute for Mathematical Sciences, The University of Edinburgh, Edinburgh, UK. g.kinnear@ed.ac.uk.
Behavior research methods
|July 10, 2025
概括
对于比较判断 (CJ) 研究,收集每个代表的十个比较是足够的. 尺度分离可靠性 (SSR) 可以可靠地估计分级间的可靠性,推值为0.8.8.
科学领域:
- 心理测量 心理测量 心理测量
- 数据分析 数据分析
- 测量理论 测量理论
背景情况:
- 比较判断 (CJ) 是一种通过对对进行比较来创建测量尺度的方法.
- 在CJ研究中,研究人员需要关于样本大小和可靠性估计 (SSR和SHR) 的指导.
- 现有的指导缺乏广泛的经验支持,并且往往是特定于学科的.
研究的目的:
- 为CJ研究中的比较数量和可靠性值提供基于证据的建议.
- 分析各种学科的大量CJ研究数据集.
- 为使用CJ方法的研究人员提供实际指导.
主要方法:
- 分析了来自不同研究领域的101个比较判断数据集.
- 统计检查比较次数与尺度可靠性之间的关系.
- 评价尺度分离可靠性 (SSR) 作为评级者间可靠性的代理.
主要成果:
- 收集每个表示的十个比较通常足以进行强大的规模构建.
- 尺度分离可靠性 (SSR) 有效地估计了评级者之间的可靠性.
- 为了提高可靠性,建议在当前的0.7标准上提高0.8的SSR值.
结论:
- 该研究为CJ研究实践提供了最新的,经验支持的指导方针.
- 研究人员可以自信地使用每项10次比较和0.8.8的SSR值.
- 这些发现增强了CJ方法在各学科的严谨性和适用性.
相关概念视频
Reliability and Validity
13.2K
Reliability and validity are two important considerations that must be made with any type of data collection. Reliability refers to the ability to consistently produce a given result. In the context of psychological research, this would mean that any instruments or tools used to collect data do so in consistent, reproducible ways.
13.2K
Statistical Analysis: Overview
7.4K
When we take repeated measurements on the same or replicated samples, we will observe inconsistencies in the magnitude. These inconsistencies are called errors. To categorize and characterize these results and their errors, the researcher can use statistical analysis to determine the quality of the measurements and/or suitability of the methods.
One of the most commonly used statistical quantifiers is the mean, which is the ratio between the sum of the numerical values of all results and the...
One of the most commonly used statistical quantifiers is the mean, which is the ratio between the sum of the numerical values of all results and the...
7.4K
Multiple Comparison Tests
4.0K
Multiple comparison test, abbreviated as MCT, is a post hoc analysis generally performed after comparing multiple samples with one or more tests. An MCT will help identify a significantly different sample among multiple samples or a factor among multiple factors.
It would be easy to compare two samples using a significance alpha level of 0.05. In other words, there is only one sample pair to be compared. However, it would be difficult to identify a significantly different sample if the number...
It would be easy to compare two samples using a significance alpha level of 0.05. In other words, there is only one sample pair to be compared. However, it would be difficult to identify a significantly different sample if the number...
4.0K
Types of Biopharmaceutical Studies: Controlled and Non-Controlled Approaches
180
Biopharmaceutical studies constitute a vital field aiming to enhance drug delivery methods and refine therapeutic approaches, drawing upon diverse interdisciplinary knowledge. In research methodologies, the choice between controlled and non-controlled studies significantly influences the study's reliability and accuracy.
Non-controlled studies, commonly employed for initial exploration, lack a control group, rendering them susceptible to biases and external influences. In contrast,...
Non-controlled studies, commonly employed for initial exploration, lack a control group, rendering them susceptible to biases and external influences. In contrast,...
180
Friedman Two-way Analysis of Variance by Ranks
306
Friedman's Two-Way Analysis of Variance by Ranks is a nonparametric test designed to identify differences across multiple test attempts when traditional assumptions of normality and equal variances do not apply. Unlike conventional ANOVA, which requires normally distributed data with equal variances, Friedman's test is ideal for ordinal or non-normally distributed data, making it particularly useful for analyzing dependent samples, such as matched subjects over time or repeated measures...
306
Statistical Methods to Analyze Parametric Data: Student t-Test and Goodness-of-Fit Test
1.8K
In parametric statistics, two fundamental tests stand out for their utility and wide application: the Student's t-test and goodness-of-fit tests. These tests provide researchers with a robust method for drawing insights from data, testing hypotheses, and making informed decisions based on their findings.
The Student's t-test is a statistical test that examines if there is a statistically significant difference between the means of two groups. This test is instrumental when dealing with...
The Student's t-test is a statistical test that examines if there is a statistically significant difference between the means of two groups. This test is instrumental when dealing with...
1.8K


