稀少是否重要? 稀少是否重要? 检查一般化理论和多面拉希测量在稀缺评级设计中的使用
Stefanie A Wind1, Eli Jones2, Sara Grajeda3
1The University of Alabama, Tuscaloosa, AL, USA.
Applied psychological measurement
|October 9, 2023
概括
与通用性 (G) 理论相比,多面拉什 (MFR) 测量更容易在稀疏的评级设计中识别评级者效应. 本模拟研究提供了分析评级者在实际评估中的表现的见解.
科学领域:
- 心理测量 心理测量 心理测量
- 教育测量教育的测量
- 统计建模 统计建模
背景情况:
- 稀疏的评级设计在绩效评估中很常见,但在这些设计中分析评级者效应是具有挑战性的.
- 现有的研究经常使用非模拟数据,限制了对评分器效应和数据不完整性进行控制的能力.
研究的目的:
- 为了比较通用性理论 (G理论) 和多面拉什 (MFR) 测量在稀疏的评级设计中检测评级者效应的有效性.
- 用模拟数据评估这些方法,以更好地了解不完整数据的影响.
主要方法:
- 进行了一项模拟研究,以生成具有受控评分器效应的绩效评估数据.
- 两个分析方法,G理论和MFR测量,应用于模拟的稀疏数据.
- 评估了每个方法识别评级者效应的能力,包括中心性和偏见.
主要成果:
- 无论是G理论还是MFR测量,都为稀疏设计中的评级质量提供了有价值的信息.
- 与G理论相比,MFR测量方法表现出更强的能力来检测评分器效应,特别是中心性和偏差.
- 模拟数据允许在数据稀疏的条件下对评分器效应进行可靠的检查.
结论:
- 在稀疏的评级设计中,MFR测量是识别特定评级器效应的更敏感的工具.
- 调查结果表明,在不完整数据的实际绩效评估中,MFR测量可能更适合进行详细的评级者质量分析.
- 进一步的研究可以建立在模拟数据方法的基础上,在复杂的评估设计中探索心理特征.
更多相关视频
09:00Author Spotlight: Validation of SICOLE-R for Assessing Cognitive and Reading Skills in Spanish-Speaking Children and Its Role in Personalized Education
Published on: August 16, 2024
801
08:12A Psychophysics Paradigm for the Collection and Analysis of Similarity Judgments
Published on: March 1, 2022
2.5K
相关概念视频
02:01Factorial Design
13.0K
Factorial Analysis is an experimental design that applies Analysis of Variance (ANOVA) statistical procedures to examine a change in a dependent variable due to more than one independent variable, also known as factors. Changes in worker productivity can be reasoned, for example, to be influenced by salary and other conditions, such as skill level. One way to test this hypothesis is by categorizing salary into three levels (low, moderate, and high) and skills sets into two levels (entry level...
13.0K
01:21Friedman Two-way Analysis of Variance by Ranks
213
Friedman's Two-Way Analysis of Variance by Ranks is a nonparametric test designed to identify differences across multiple test attempts when traditional assumptions of normality and equal variances do not apply. Unlike conventional ANOVA, which requires normally distributed data with equal variances, Friedman's test is ideal for ordinal or non-normally distributed data, making it particularly useful for analyzing dependent samples, such as matched subjects over time or repeated measures...
213
01:20Spearman's Rank Correlation Test
826
Spearman's rank correlation test, also known as Spearman's rho, is a nonparametric method for assessing the strength and direction of association between two variables. This test is particularly valuable when the data distribution is unknown or when the assumption of normality does not hold. Named after the English psychologist and statistician Dr. Charles Edward Spearman, it serves as the nonparametric counterpart to Pearson's correlation coefficient.
Spearman's test calculates...
Spearman's test calculates...
826
01:29Reliability and Validity
12.7K
Reliability and validity are two important considerations that must be made with any type of data collection. Reliability refers to the ability to consistently produce a given result. In the context of psychological research, this would mean that any instruments or tools used to collect data do so in consistent, reproducible ways.
12.7K
01:16Response Surface Methodology
157
Response Surface Methodology (RSM) is a collection of statistical and mathematical techniques used to develop, improve, and optimize processes. It is particularly valuable when many input variables or factors potentially influence a response variable.
The process of RSM involves several key steps:
The process of RSM involves several key steps:
157
01:52Regression Toward the Mean
6.3K
Regression toward the mean (“RTM”) is a phenomenon in which extremely high or low values—for example, and individual’s blood pressure at a particular moment—appear closer to a group’s average upon remeasuring. Although this statistical peculiarity is the result of random error and chance, it has been problematic across various medical, scientific, financial and psychological applications. In particular, RTM, if not taken into account, can interfere when...
6.3K