使用项目响应理论评估和跟踪定性内容编码器性能
Michael Hennessy1, Amy Bleakley1, Morgan E Ellithorpe1
1University of Delaware, Department of Communication, 250 Pearson Hall, Newark, DE 19716.
Quality & quantity
|December 4, 2023
概括
项目响应理论 (IRT) 提供了一种新方法来跟踪媒体内容分析中的个人编码器性能,提供了超越传统协议措施的洞察力. 这种方法通过详细说明编码器随时间变化的一致性,提高了内容编码的可靠性.
科学领域:
- 沟通研究 沟通研究
- 媒体研究 媒体研究
- 量化心理学 量化心理学
背景情况:
- 内容分析对于理解媒体内容,曝光和跨传统和社交媒体平台的影响至关重要.
- 传统的可靠性指标,如卡帕式统计,总结了编码器的决策,掩盖了个别编码器的性能和一致性.
- 准确评估个别编码器的可靠性对于内容分析结果的有效性至关重要.
研究的目的:
- 引入和评估项目响应理论 (IRT) 作为一种用于评估内容分析中个别编码器性能的新方法.
- 为了证明IRT在随着时间的推移和不同的编码项目中跟踪编码器一致性的能力.
- 在媒体内容分析中为聚合可靠性测量提供编码器特定的替代方案.
主要方法:
- 使用了一个包含105个体育和能量饮料广告的数据集.
- 雇佣了五名人类编码员,在选定的媒体项目上进行定性内容编码.
- 应用物件响应理论 (IRT) 模型来分析编码数据并评估个别编码器的性能.
主要成果:
- 项目响应理论 (IRT) 在研究期间成功跟踪了个别编码者的表现.
- IRT提供了关于每个编码器对定性编码对象的决定一致性的具体见解.
- 结果表明,IRT提供了与总量措施相比,对编码器可靠性的更细致的理解.
结论:
- 项目响应理论 (IRT) 是内容分析师方法论工具包中的一个有价值的补充.
- 通过IRT,可以精确地测量和跟踪个人内容编码器的性能和决策一致性.
- 采用IRT可以提高媒体内容分析中的可靠性评估的严谨性和透明度.
相关概念视频
Cochran's Q Test
353
Cochran's Q Test is a nonparametric statistical test used to determine if there are potential differences in the outcomes of three or more related groups on a binary (yes/no) or dichotomous outcome. It is essentially an extension of the McNemar Test, which is limited to two related samples - Cochran's Q test can handle three or more related samples, making it more versatile in scenarios where subjects are measured under multiple conditions. The test statistic follows a Chi-Square...
353
Qualitative Analysis
22.4K
For solutions containing mixtures of different cations, the identity of each cation can be determined by qualitative analysis. This technique involves a series of selective precipitations with different chemical reagents, each reaction producing a characteristic precipitate for a specific group of cations. Metal ions within a group are further separated by varying the pH, heating the mixture to redissolve a precipitate, or adding other reagents to form complex ions.
For instance, group IV...
For instance, group IV...
22.4K
Reaction Quotient
48.6K
The status of a reversible reaction is conveniently assessed by evaluating its reaction quotient (Q). For a reversible reaction described by m A + n B ⇌ x C + y D, the reaction quotient is derived directly from the stoichiometry of the balanced equation as
48.6K
Goodness-of-Fit Test
3.4K
The goodness-of-fit test is a type of hypothesis test which determines whether the data "fits" a particular distribution. For example, one may suspect that some anonymous data may fit a binomial distribution. A chi-square test (meaning the distribution for the hypothesis test is chi-square) can be used to determine if there is a fit. The null and alternative hypotheses may be written in sentences or stated as equations or inequalities. The test statistic for a goodness-of-fit test is given as...
3.4K
Response Surface Methodology
140
Response Surface Methodology (RSM) is a collection of statistical and mathematical techniques used to develop, improve, and optimize processes. It is particularly valuable when many input variables or factors potentially influence a response variable.
The process of RSM involves several key steps:
The process of RSM involves several key steps:
140
Self-Evaluation: Self-Enhancement and Self-Verification
5.2K
Social psychologists have documented that feeling good about ourselves and maintaining positive self-esteem is a powerful motivator of human behavior (Tavris & Aronson, 2008). In the United States, members of the predominant culture typically think very highly of themselves and view themselves as good people who are above average on many desirable traits (Ehrlinger, Gilovich, & Ross, 2005). Often, our behavior, attitudes, and beliefs are affected when we experience a threat to our...
5.2K


