Related Experiment Video
Updated: Aug 22, 2025
08:48
Combining Multiple Data Acquisition Systems to Study Corticospinal Output and Multi-segment Biomechanics
Published on: January 9, 2016
7.0K
Too Good to Be True: Bots and Bad Data From Mechanical Turk
Margaret A Webb1,2, June P Tangney2
1Department of Criminology, Max-Planck Institute for the Study of Crime, Security, and Law Freiburg im Breisgau.
Summary
Researchers caution against using Amazon
Area of Science:
- Behavioral and social sciences research
- Online data collection methodologies
Background:
- Psychology increasingly relies on digital data sources.
- Amazon's Mechanical Turk (MTurk) is a prominent platform for online studies.
- In 2015, an estimated 45% of top behavioral and social science journals featured MTurk studies.
Purpose of the Study:
- To report on the validity of data collected via Amazon's Mechanical Turk (MTurk).
- To serve as a cautionary example for researchers utilizing MTurk for data collection.
Main Methods:
- An online study was conducted using Amazon's Mechanical Turk (MTurk).
- Data validity was assessed to determine the proportion of reliable participant responses.
- Participant sample size was N = 529.
Main Results:
- An estimated 2.6% of the participant sample (14 out of 529 individuals) provided valid data.
- The overall data validity was significantly lower than anticipated.
Conclusions:
- The findings suggest that data collected through MTurk may not always be reliable.
- Researchers are urged to exercise caution and implement rigorous validation checks when using MTurk for data collection.
- The platform's efficiency in data gathering may be offset by concerns regarding data quality.
More Related Videos
Related Concept Videos
01:52Regression Toward the Mean
6.3K
Regression toward the mean (“RTM”) is a phenomenon in which extremely high or low values—for example, and individual’s blood pressure at a particular moment—appear closer to a group’s average upon remeasuring. Although this statistical peculiarity is the result of random error and chance, it has been problematic across various medical, scientific, financial and psychological applications. In particular, RTM, if not taken into account, can interfere when...
6.3K
01:14Mechanical Efficiency of Real Machines
808
The mechanical efficiency of a machine is a fundamental concept that describes how effectively a machine can convert input work into output work. According to this concept, the efficiency of a machine is equal to the ratio of the output work to the input work. An ideal machine, meaning a machine that has no energy losses, has an efficiency of one. This implies that the input work and the output work are equal.
However, in reality, no machine can be truly ideal, and all of them experience some...
However, in reality, no machine can be truly ideal, and all of them experience some...
808
02:16Stereotype Content Model
14.8K
The Stereotype Content Model (SCM) was first proposed by Susan Fiske and her colleagues (Fiske, Cuddy, Glick & Xu, 2002; see also Fiske, 2012 and Fiske, 2017). The SCM specifies that when someone encounters a new group, they will stereotype them based on two metrics: warmth—or that group’s perceived intent, and how likely they are to provide help or inflict harm—and competence—or their ability to carry out that objective. Depending on the warmth-competence...
14.8K