评估中心的评级反映了什么? 跨多个样本的差异组成中的一致性和异质性
Pia V Ingold1, Anna Luca Heimann2, Bettina Waller3
1Department of Psychology, University of Copenhagen.
The Journal of applied psychology
|September 29, 2025
概括
这项研究揭示了评估对象及其与练习和维度的相互作用是评估中心 (AC) 中最大的差异来源. 聚合得分稍微减少了异质性,突出了需要多方面的交流功能.
科学领域:
- 组织心理学 组织心理学
- 心理测量 心理测量 心理测量
背景情况:
- 评估中心 (ACs) 的测量属性受到辩论,最近的研究重点是概括性理论和分数聚合.
- 了解差异组件和评分器间可靠性对于AC有效性至关重要.
研究的目的:
- 通过使用贝叶斯概括性理论,提供第一个通过AC元件解释的方差的元分析估计.
- 检查各种AC样本中这些效应的概括性和异质性.
- 调查分数聚合对异质性的影响.
主要方法:
- 贝叶斯概括性理论和随机效应元分析应用于19个AC样本 (N = 4,963).
- 检查的差异归因于评估者,练习,维度,评估者,他们的相互作用和评级者之间的可靠性.
- 分析了将评级汇总成整体,运动级和维度级分数的分析效应.
主要成果:
- 评估者主要影响和评估者行使影响是最大的差异组件,与之前的研究一致.
- 评估主体-行使-维度的影响也很大.
- 在可靠的差异组件和跨样本的interrater可靠性中发现了实质性的异质性.
- 积分聚合最小化了异质性.
结论:
- 评估者,运动和维度的相互作用是理解AC测量的关键.
- 样本中的高异质性需要仔细考虑交流设计和背景.
- 目前的得分聚合方法在减少测量方差方面提供了有限的好处.
相关概念视频
One-Way ANOVA: Equal Sample Sizes
4.0K
One-Way ANOVA can be performed on three or more samples with equal or unequal sample sizes. When one-way ANOVA is performed on two datasets with samples of equal sizes, it can be easily observed that the computed F statistic is highly sensitive to the sample mean.
Different sample means can result in different values for the variance estimate: variance between samples. This is because the variance between samples is calculated as the product of the sample size and the variance between the...
Different sample means can result in different values for the variance estimate: variance between samples. This is because the variance between samples is calculated as the product of the sample size and the variance between the...
4.0K
Variability: Analysis
448
Measures of variability are statistical metrics that reveal the dispersion pattern within a dataset. They are pivotal in biostatistics, providing insights into the heterogeneity within health and biological data. Variability signifies the degree to which data points diverge from one another, helping researchers understand the potential range of values and associated uncertainty within the data.
The range is a simple measure of variability, indicating the difference between the highest and...
The range is a simple measure of variability, indicating the difference between the highest and...
448
What is Variation?
17.6K
Apart from the measures of central tendency, distribution, outliers, and the changing characteristics of data with time, an important characteristic of any data set is its variation or spread. In some data sets, the data values are concentrated closely near the mean; in others, the data values are more widely spread out from the mean.
The range, standard deviation, standard error, and variance are the different measures of variation.
Range: The range is the difference between its maximum and...
The range, standard deviation, standard error, and variance are the different measures of variation.
Range: The range is the difference between its maximum and...
17.6K
One-Way ANOVA
11.9K
One-way ANOVA analyzes more than three samples categorized by one factor. For example, it can compare the average mileage of sports bikes. Here, the data is categorized by one factor - the company. However, one-way ANOVA cannot be used to simultaneously compare the sample mean of three or more samples categorized by two factors. An example of two factors would be sports bikes from different companies driven in different terrains, such as a desert or snowy landscape. Here, two-way ANOVA is used...
11.9K
Statistical Analysis: Overview
14.6K
When we take repeated measurements on the same or replicated samples, we will observe inconsistencies in the magnitude. These inconsistencies are called errors. To categorize and characterize these results and their errors, the researcher can use statistical analysis to determine the quality of the measurements and/or suitability of the methods.
One of the most commonly used statistical quantifiers is the mean, which is the ratio between the sum of the numerical values of all results and the...
One of the most commonly used statistical quantifiers is the mean, which is the ratio between the sum of the numerical values of all results and the...
14.6K
Review and Preview
8.3K
In statistics, several tools are used to interpret the data. Measures of central tendency represent the characteristics of the data, such as mean, median, and mode. Additionally, measures of variance like standard deviation and range are used to find the spread of data from the mean. Relative standing measures the distance between data locations. Commonly used measures of relative standings are percentile, z score, and quartiles.
Percentiles are a type of fractile that partition data into...
Percentiles are a type of fractile that partition data into...
8.3K


