使用猜测模型和知识系数的测量协议
1Department of Data Science and Analytics, BI Norwegian Business School, Oslo, Norway. jonas.moss@bi.no.
Psychometrika
|June 8, 2023
概括
这项研究介绍了猜测模型和知识系数,用于测量评价者之间的协议. 布伦南-普雷迪格系数在模拟中表现出卓越的性能,在具有挑战性的条件下提供更好的覆盖范围.
科学领域:
- 统计 统计 统计 统计
- 心理测量 心理测量 心理测量
- 数据分析 数据分析
背景情况:
- 像科恩的卡帕和弗莱斯的卡帕这样的现有协议措施依赖于特定的评级模型.
- 需要一个统一的框架来涵盖法官评分行为的各种明确模型.
研究的目的:
- 提出一种一般类型的模型,称为"猜测模型",用于理解法官如何进行评级.
- 引入与这些模型相关的统一的协议度量,即"知识系数".
- 评估不同协议措施的表现,包括建议的系数,在不同的条件下.
主要方法:
- 开发"猜测模型"框架,以将现有评级模型通用化.
- 导出"知识系数"作为在这个框架内达成协议的衡量标准.
- 估计知识系数,并分析其非对称分布.
- 进行灵敏度分析和模拟研究,以比较信任区间覆盖范围.
主要成果:
- 知识系数在特定假设下统一了多项现有协议措施.
- 发现布伦南-普雷迪格系数是知识系数的可靠估计器.
- 模拟结果表明,布伦南-普雷迪格系数提供了优越的置信区间覆盖率,特别是在不利的情况下.
结论:
- 估计模型为协议分析提供了灵活的框架.
- 推使用布伦南-普雷迪格系数,因为它具有强大的性能,并且在评估者之间的可靠性研究中覆盖率更高.
- 知识系数为衡量协议提供了一种统一的方法,可以适应不同的法官评级模型.
相关概念视频
One-Compartment Open Model: Wagner-Nelson and Loo Riegelman Method for ka Estimation
589
This lesson introduces two critical methods in pharmacokinetics, the Wagner-Nelson and Loo-Riegelman methods, used for estimating the absorption rate constant (ka) for drugs administered via non-intravenous routes. The Wagner-Nelson method relates ka to the plasma concentration derived from the slope of a semilog percent unabsorbed time plot. However, it is limited to drugs with one-compartment kinetics and can be impacted by factors like gastrointestinal motility or enzymatic degradation.
On...
On...
589
Accuracy, limits, and approximation
484
Accuracy, limits, and approximations are common in many fields, especially in engineering calculations. These concepts are imperative for ensuring that a given value is as close as possible to its true value.
Accuracy is defined as the closeness of the measured value to the true or actual value. In engineering mechanics, repeated measurements are taken during theoretical or experimental analyses to ensure that the result is precise and accurate.
The accuracy of any solution is based on the...
Accuracy is defined as the closeness of the measured value to the true or actual value. In engineering mechanics, repeated measurements are taken during theoretical or experimental analyses to ensure that the result is precise and accurate.
The accuracy of any solution is based on the...
484
Accuracy and Errors in Hypothesis Testing
232
Hypothesis testing is a fundamental statistical tool that begins with the assumption that the null hypothesis H0 is true. During this process, two types of errors can occur: Type I and Type II. A Type I error refers to the incorrect rejection of a true null hypothesis, while a Type II error involves the failure to reject a false null hypothesis.
In hypothesis testing, the probability of making a Type I error, denoted as α, is commonly set at 0.05. This significance level indicates a 5%...
In hypothesis testing, the probability of making a Type I error, denoted as α, is commonly set at 0.05. This significance level indicates a 5%...
232
Estimating Population Mean with Unknown Standard Deviation
8.2K
In practice, we rarely know the population standard deviation. In the past, when the sample size was large, this did not present a problem to statisticians. They used the sample standard deviation s as an estimate for σ and proceeded as before to calculate a confidence interval with close enough results. However, statisticians ran into problems when the sample size was small. A small sample size caused inaccuracies in the confidence interval.
William S. Gosset (1876–1937) of the...
William S. Gosset (1876–1937) of the...
8.2K
Estimating Population Mean with Known Standard Deviation
8.9K
To construct a confidence interval for a single unknown population mean μ, where the population standard deviation is known, we need sample mean as an estimate for μ and we need the margin of error. Here, the margin of error (EBM) is called the error bound for a population mean (abbreviated EBM). The sample mean is the point estimate of the unknown population mean μ.
The confidence interval estimate will have the form as follows:
(point estimate - error bound, point estimate +...
The confidence interval estimate will have the form as follows:
(point estimate - error bound, point estimate +...
8.9K
Mechanistic Models: Compartment Models in Individual and Population Analysis
67
Mechanistic models are utilized in individual analysis using single-source data, but imperfections arise due to data collection errors, preventing perfect prediction of observed data. The mathematical equation involves known values (Xi), observed concentrations (Ci), measurement errors (εi), model parameters (ϕj), and the related function (ƒi) for i number of values. Different least-squares metrics quantify differences between predicted and observed values. The ordinary least...
67


