Related Experiment Videos
Conducting Online Behavioral Research Using Crowdsourcing Services in Japan
Yoshimasa Majima1, Kaoru Nishiyama1, Aki Nishihara2
1Department of Psychology for Well-Being, Hokusei Gakuen University Sapporo, Japan.
Frontiers in Psychology
|April 7, 2017
Summary
This study validates Japanese crowdsourcing workers for behavioral research, finding them comparable to traditional samples. Small screen devices negatively impact attention to instructions, suggesting device-specific recommendations are needed.
Area of Science:
- Psychology
- Human Behavior Research
- Crowdsourcing Studies
Background:
- Online labor markets, or crowdsourcing, are increasingly used for human behavior research.
- Amazon's Mechanical Turk (MTurk) is a dominant platform, with prior studies validating its worker pool against traditional samples.
- Existing research has not extensively validated non-MTurk, ethnically diverse crowdsourcing samples.
Purpose of the Study:
- To extend validation findings to non-MTurk crowdsourcing samples, specifically Japanese workers.
- To compare Japanese crowdsourcing workers with university students on key research metrics.
- To identify factors influencing data quality in diverse online samples.
Main Methods:
- Conducted three surveys with Japanese crowdsourcing workers and university students (N=1046 total).
- Assessed demographics, personality traits, reasoning skills, and attention to instructions.
- Utilized comparative analysis to evaluate sample eligibility for behavioral research.
Main Results:
- Japanese crowdsourcing participants demonstrated comparability to traditional samples in demographics, personality, and reasoning.
- Attention to instructions was negatively impacted by the use of small screen devices.
- Findings support the eligibility of non-MTurk crowdsourcing samples for behavioral research.
Conclusions:
- Non-MTurk crowdsourcing samples, including Japanese workers, are suitable for behavioral research.
- Researchers should consider device type when designing online studies to ensure data quality.
- Recommendations are provided for optimizing data collection with diverse online participant pools.