Related Experiment Video
Updated: Aug 23, 2025

Methods to Test Visual Attention Online
Published on: February 19, 2015
Comparing Data Collected on Amazon's Mechanical Turk to National Surveys
Nabeel Qureshi1, Maria Edelen2, Lara Hilton3
1RAND Corporation, Santa Monica, CA, United States.
Objective:
In this study, we examined the impact of a range of methods to improve data quality on the demographic and health status representativeness of Amazon Mechanical Turk (MTurk) samples.
Methods:
We developed and field-tested a general survey of health on MTurk in 2017 among 5755 participants and 2021 among 6752 participants. We collected information on participant demographic characteristics and health status and implemented different quality checks in 2017 and 2021.
Results:
Adding data quality checks generally improves the representativeness of the final MTurk sample, but there are persistent differences in mental health and pain conditions, age, education, and income between the MTurk population and the broader US population.
Conclusion:
We conclude that data quality checks improve the data quality and representativeness.
More Related Videos
Related Concept Videos
Surveys
Data Collection by Survey
Regression Toward the Mean
Types of Surveys
Measures of Central Tendency
Comparing Experimental Results: Student's t-Test

