评估2024年亚马逊机械土耳其 (MTurk) 数据的质量和可靠性
Hagar Shimoni1, Vadim Axelrod1
1The Gonda Multidisciplinary Brain Research Center, Bar-Ilan University, Ramat Gan, Israel.
Royal Society open science
|July 29, 2025
概括
亚马逊机械土耳其 (MTurk) 主工提供可靠的数据用于心理研究,与其他工人群体不同. 这项研究建议使用MTurk主处理器在在线研究中实现最佳数据质量.
科学领域:
- 心理学 心理学 心理学
- 社会科学 社会科学 社会科学
- 人与计算机的交互
背景情况:
- 亚马逊机械土耳其人 (MTurk) 是一个受欢迎的在线心理学和社会科学研究平台.
- 有关数据质量和MTurk用于研究目的的最佳使用存在担忧.
研究的目的:
- 系统地调查亚马逊机械土耳其 (MTurk) 工人的数据质量和可靠性.
- 为了确定不同MTurk工作者队伍的适合性进行研究.
主要方法:
- 在1300多名参与者中进行了主要和复制实验.
- 参与者被分为三个群体:MTurk Master 工作人员,非Master 工作人员和非Master 工作人员,认可率>=95%.
- 评估了注意力检查,可靠性和直线趋势.
主要成果:
- MTurk Master工人表现出高可靠性,很少错过注意力检查,并且没有表现出直线性.
- 非主人的工人,包括那些有95%的批准率的工人,经常错过检查,显示出低可靠性,并倾向于直线.
- 人们对非Master MTurk员工的适合性提出了重大担忧.
结论:
- 推MTurk Master工人用于研究,特别是当参与者的天真性不至关重要并且不需要大样本时.
- 来自非Master MTurk员工的数据质量引发了严重的担忧,质疑它们是否适合可靠的研究.
- 优化使用MTurk进行研究需要根据数据质量要求仔细选择工作人员队列.
相关概念视频
Reliability and Validity
13.2K
Reliability and validity are two important considerations that must be made with any type of data collection. Reliability refers to the ability to consistently produce a given result. In the context of psychological research, this would mean that any instruments or tools used to collect data do so in consistent, reproducible ways.
13.2K
Regression Toward the Mean
6.5K
Regression toward the mean (“RTM”) is a phenomenon in which extremely high or low values—for example, and individual’s blood pressure at a particular moment—appear closer to a group’s average upon remeasuring. Although this statistical peculiarity is the result of random error and chance, it has been problematic across various medical, scientific, financial and psychological applications. In particular, RTM, if not taken into account, can interfere when...
6.5K
Distribution Reliability and Automation
162
Distribution reliability in electrical power systems is critical for ensuring an uninterrupted power supply to consumers at minimal cost. According to IEEE Standard Terms, reliability is the probability that a device will function without failure over a specified time period or amount of usage. For electric power distribution, this translates to maintaining continuous power supply and addressing customer concerns over power outages. Several indices, as defined by IEEE Standard 1366-2012, are...
162
Improving Translational Accuracy
2.7K
2.7K
Mechanical Efficiency of Real Machines
864
The mechanical efficiency of a machine is a fundamental concept that describes how effectively a machine can convert input work into output work. According to this concept, the efficiency of a machine is equal to the ratio of the output work to the input work. An ideal machine, meaning a machine that has no energy losses, has an efficiency of one. This implies that the input work and the output work are equal.
However, in reality, no machine can be truly ideal, and all of them experience some...
However, in reality, no machine can be truly ideal, and all of them experience some...
864


