基于评分者的评估不一致性主要影响边界候选人:但使用简单的启发式可能会改善通过-失败决策
Stefan K Schauber1,2, Anne O Olsen3, Erik L Werner4
1Centre for Health Sciences Education, Faculty of Medicine, University of Oslo, Oslo, Norway. stefanks@uio.no.
Advances in health sciences education : theory and practice
|April 22, 2024
概括
评估中的专家判断是不一致的,特别是在表现较低的情况下. 简单的策略可以改善教育评估中的评级者一致性和决策.
科学领域:
- 医学教育 医学教育
- 心理学 心理学 心理学
- 认知科学 认知科学
背景情况:
- 专家判断在各个领域显示出显著的不一致性.
- 在医学教育中,检查员在评估中的分歧可能会产生严重后果.
- 现有文献表明,评级不一致可能与候选人的绩效水平相关,这一领域需要进一步的统计调查.
研究的目的:
- 调查评级人员一致性与考生在评估中的绩效水平之间的关系.
- 探索简单启发式的实用性,以提高考官之间的决策.
主要方法:
- 记录了用于分析的模拟检查.
- 十名考官评估了四名学生的表现.
- 启动程序被用来比较考官对之间的绩效评级不一致性.
主要成果:
- 观察到评级差异与候选人的绩效水平有很大的依赖.
- 与表现出色的候选人相比,表现较差的候选人获得了更多不一致的评分.
- 分析表明,简单的启发式分析可能会改善考官对中的决策.
结论:
- 证实了评级者不一致性和考生在评估中的表现之间的联系.
- 强调考虑绩效水平的重要性,以解决判断的变化.
- 倡导采用生态理性原则和简单的启发式方法来改善评估实践和未来的研究方向.
相关概念视频
Reliability and Validity
12.7K
Reliability and validity are two important considerations that must be made with any type of data collection. Reliability refers to the ability to consistently produce a given result. In the context of psychological research, this would mean that any instruments or tools used to collect data do so in consistent, reproducible ways.
12.7K
Self-Discrepancy Theory
18.3K
One influential perspective on what motivates people's behavior is detailed in Tory Higgin's self-discrepancy theory (Higgins, 1987). He proposed that people hold disagreeing internal representations of themselves that lead to different emotional states.
18.3K
The Availability Heuristic
6.0K
A heuristic is a general problem-solving framework (Tversky & Kahneman, 1974). You can think of these as mental shortcuts that are used to solve problems. Different types of heuristics are used in different types of situations, and the impulse to use a heuristic occurs when one of five conditions is met (Pratkanis, 1989):
6.0K
Heuristics
84
Heuristics are problem-solving strategies that use mental shortcuts to simplify decision-making. Unlike algorithms, which must be followed precisely to achieve a correct result, heuristics offer a general problem-solving framework. They save time and energy but can sometimes lead to less rational decisions.
People often rely on heuristics when faced with an overload of information, limited time, low importance of the decision, limited information, or when a heuristic readily comes to mind. For...
People often rely on heuristics when faced with an overload of information, limited time, low importance of the decision, limited information, or when a heuristic readily comes to mind. For...
84
Fundamental Attribution Error
12.8K
According to some social psychologists, people tend to overemphasize internal factors as explanations—or attributions—for the behavior of other people. They tend to assume that the behavior of another person is a trait of that person, and to underestimate the power of the situation on the behavior of others. They tend to fail to recognize when the behavior of another is due to situational variables, and thus to the person’s state. This erroneous assumption is...
12.8K
Confirmation Biases
5.5K
The confirmation bias is the tendency to focus on information that confirms our existing beliefs and ignore information that is inconsistent with our expectations. For example, if you think that your professor is not very nice, you notice all of the instances of rude behavior exhibited by the professor while ignoring the countless pleasant interactions he is involved in on a daily basis. Have you ever fallen prey to the confirmation bias, either as the source or target of such bias?
5.5K


