一本全面的指南,研究多观察者顺序数据的一致性和可靠性
Sophie Vanbelle1, Christina Hernandez Engelhart2,3, Ellen Blix3
1Methodology and Statistics, CAPHRI, Maastricht University, P. Debyeplein, 1, Maastricht, 6229 HA, The Netherlands. sophie.vanbelle@maastrichtuniversity.nl.
BMC medical research methodology
|December 20, 2024
概括
本研究提供了选择和规划协议和可靠性研究的指导,用于多个观察员的顺序结果. 它引入了方法和开源软件,以提高临床试验可靠性研究的质量.
科学领域:
- 统计 统计 统计 统计
- 生物统计学 生物统计学
- 心理测量 心理测量 心理测量
背景情况:
- 系统性审查突出了对顺序尺度的一致性和可靠性研究的问题,特别是在多个观察者的情况下.
- 现有的指导方针需要加强,以便全面报告和执行此类研究.
研究的目的:
- 提供全面的信息,选择和规划协议和可靠性研究的顺序结果.
- 为了应对涉及两个以上观察员的研究中的挑战.
主要方法:
- 对多个观察者有序结果的协议措施 (同意/不同意的比例,平均绝对偏差,平均平方偏差,加权卡帕) 的概括.
- 使用德尔塔方法进行大样本方差估计,以构建瓦尔德置信区间.
- 制定一项程序,以确定所需的最低数量的评分员和患者,以尽量减少采样不确定性.
主要成果:
- 提供了对协议和可靠性系数的明确解释,区分了这两个概念.
- 对各种系数的样本方差进行呈现或推导,从而使沃尔德置信区间的构造成为可能.
- 为控制不确定性,建立了样本大小确定 (评分者和患者) 的程序.
结论:
- 该论文补充了现有的指导方针,如GRRAS,以提高临床试验可靠性和协议研究的质量.
- 提供开源软件 (R包和Shiny应用程序),以方便研究人员应用这些方法.
相关概念视频
Ordinal Level of Measurement
22.9K
The way a set of data is measured is called its level of measurement. Correct statistical procedures depend on a researcher being familiar with levels of measurement. For analysis, data are classified into four levels of measurement—nominal, ordinal, interval, and ratio.
Data measured using an ordinal scale are similar to nominal scale data, but there is one major difference. The ordinal scale data can be ordered. An example of ordinal scale data is a list of the top five national parks...
Data measured using an ordinal scale are similar to nominal scale data, but there is one major difference. The ordinal scale data can be ordered. An example of ordinal scale data is a list of the top five national parks...
22.9K
Reliability and Validity
12.7K
Reliability and validity are two important considerations that must be made with any type of data collection. Reliability refers to the ability to consistently produce a given result. In the context of psychological research, this would mean that any instruments or tools used to collect data do so in consistent, reproducible ways.
12.7K
Kendall's Coefficient of Concordance
246
Kendall's Coefficient of Concordance (W), also known as Kendall's W, is a non-parametric statistical measure used to assess the agreement or concordance between multiple raters or judges when they rank a set of items. It is often used when you have ordinal data (ranks) and you want to see if there is consistency or consensus among the raters. It is widely applied in research areas such as psychology, medicine, and social sciences, where multiple judges are asked to rank or rate subjects...
246
Friedman Two-way Analysis of Variance by Ranks
144
Friedman's Two-Way Analysis of Variance by Ranks is a nonparametric test designed to identify differences across multiple test attempts when traditional assumptions of normality and equal variances do not apply. Unlike conventional ANOVA, which requires normally distributed data with equal variances, Friedman's test is ideal for ordinal or non-normally distributed data, making it particularly useful for analyzing dependent samples, such as matched subjects over time or repeated measures...
144
Statistical Analysis: Overview
6.1K
When we take repeated measurements on the same or replicated samples, we will observe inconsistencies in the magnitude. These inconsistencies are called errors. To categorize and characterize these results and their errors, the researcher can use statistical analysis to determine the quality of the measurements and/or suitability of the methods.
One of the most commonly used statistical quantifiers is the mean, which is the ratio between the sum of the numerical values of all results and the...
One of the most commonly used statistical quantifiers is the mean, which is the ratio between the sum of the numerical values of all results and the...
6.1K
Naturalistic Observations
15.4K
If you want to understand how behavior occurs, one of the best ways to gain information is to simply observe the behavior in its natural context. However, people might change their behavior in unexpected ways if they know they are being observed. How do researchers obtain accurate information when people tend to hide their natural behavior? As an example, imagine that your professor asks everyone in your class to raise their hand if they always wash their hands after using the restroom. Chances...
15.4K


