对直接监督人员绩效评级的评级者间可靠性的元分析估计:在最佳测量设计下乐观
Andrew B Speer1, Angie Y Delacruz2, Lauren J Wegmeyer2
1Department of Management and Entrepreneurship, Kelley School of Business, Indiana University.
The Journal of applied psychology
|October 12, 2023
概括
绩效评估 (PA) 的可靠性经常被低估. 这项研究发现,直接主管评级显示出更高的评级者间可靠性 (IRR),这表明更准确的员工绩效评估.
科学领域:
- 组织心理学 组织心理学
- 人力资源管理 人力资源管理
背景情况:
- 绩效评估 (PA) 对组织至关重要,但其可靠性经常受到质疑.
- 现有的元分析可能会通过使用非并行评分器设计来低估PA的可靠性.
研究的目的:
- 重新评估绩效评估可靠性,使用更严格的评价者间可靠性 (IRR) 的定义.
- 为了确定员工直接主管提供的评级的可靠性.
主要方法:
- 进行了对22个独立样本的元分析,这些样本符合特定的纳入标准.
- 专注于评估者直接监督员工工作表现的研究.
主要成果:
- 直接监管机构评级的平均观察到的评级者间可靠性 (IRR) 为0.65.
- 可靠性在作战环境中为0.60,在研究环境中为0.67.
结论:
- 提出的元分析IRR估计是最准确的可用于直接监管机构可靠性.
- 这些发现支持在未来的研究和人力资源实践中使用直接监督者评级.
相关概念视频
01:21Friedman Two-way Analysis of Variance by Ranks
208
Friedman's Two-Way Analysis of Variance by Ranks is a nonparametric test designed to identify differences across multiple test attempts when traditional assumptions of normality and equal variances do not apply. Unlike conventional ANOVA, which requires normally distributed data with equal variances, Friedman's test is ideal for ordinal or non-normally distributed data, making it particularly useful for analyzing dependent samples, such as matched subjects over time or repeated measures...
208
01:20Spearman's Rank Correlation Test
822
Spearman's rank correlation test, also known as Spearman's rho, is a nonparametric method for assessing the strength and direction of association between two variables. This test is particularly valuable when the data distribution is unknown or when the assumption of normality does not hold. Named after the English psychologist and statistician Dr. Charles Edward Spearman, it serves as the nonparametric counterpart to Pearson's correlation coefficient.
Spearman's test calculates...
Spearman's test calculates...
822
01:20Kendall's Coefficient of Concordance
365
Kendall's Coefficient of Concordance (W), also known as Kendall's W, is a non-parametric statistical measure used to assess the agreement or concordance between multiple raters or judges when they rank a set of items. It is often used when you have ordinal data (ranks) and you want to see if there is consistency or consensus among the raters. It is widely applied in research areas such as psychology, medicine, and social sciences, where multiple judges are asked to rank or rate subjects...
365
01:22Self-Report Tests of Personality
362
Self-report inventories are objective personality assessments that use multiple-choice items or numbered scales, typically ranging from 1 (strongly disagree) to 5 (strongly agree). They are often called Likert scales after Rensis Likert. These inventories are widely used due to their ease of administration and cost-effectiveness. One of the most prominent examples is the Minnesota Multiphasic Personality Inventory (MMPI), initially developed in the 1940s to assess abnormal personality traits.
362
01:29Reliability and Validity
12.7K
Reliability and validity are two important considerations that must be made with any type of data collection. Reliability refers to the ability to consistently produce a given result. In the context of psychological research, this would mean that any instruments or tools used to collect data do so in consistent, reproducible ways.
12.7K
01:11Statistical Analysis: Overview
6.6K
When we take repeated measurements on the same or replicated samples, we will observe inconsistencies in the magnitude. These inconsistencies are called errors. To categorize and characterize these results and their errors, the researcher can use statistical analysis to determine the quality of the measurements and/or suitability of the methods.
One of the most commonly used statistical quantifiers is the mean, which is the ratio between the sum of the numerical values of all results and the...
One of the most commonly used statistical quantifiers is the mean, which is the ratio between the sum of the numerical values of all results and the...
6.6K