Jove
Visualize
Contact Us
JoVE
x logofacebook logolinkedin logoyoutube logo
ABOUT JoVE
OverviewLeadershipBlogJoVE Help Center
AUTHORS
Publishing ProcessEditorial BoardScope & PoliciesPeer ReviewFAQSubmit
LIBRARIANS
TestimonialsSubscriptionsAccessResourcesLibrary Advisory BoardFAQ
RESEARCH
JoVE JournalMethods CollectionsJoVE Encyclopedia of ExperimentsArchive
EDUCATION
JoVE CoreJoVE BusinessJoVE Science EducationJoVE Lab ManualFaculty Resource CenterFaculty Site
Terms & Conditions of Use
Privacy Policy
Policies

Related Concept Videos

Regression Toward the Mean01:52

Regression Toward the Mean

6.6K
Regression toward the mean (“RTM”) is a phenomenon in which extremely high or low values—for example, and individual’s blood pressure at a particular moment—appear closer to a group’s average upon remeasuring. Although this statistical peculiarity is the result of random error and chance, it has been problematic across various medical, scientific, financial and psychological applications. In particular, RTM, if not taken into account, can interfere when...
6.6K

You might also read

Related Articles

Articles linked to this work by shared authors, journal, and citation graph.

Sort by
Same author

The properties of panels in global comics: frequency and size of 76 K panels in 1,030 comics from 144 countries.

Language resources and evaluation·2026
Same author

The interplay between selective attention and summary statistics.

The Behavioral and brain sciences·2025
Same author

A selective sampling account of forming numerosity representations.

Psychological review·2025
Same author

Using diffusion models for symbolic numeracy tasks to examine aging effects.

Journal of experimental psychology. Learning, memory, and cognition·2024
Same author

Exploring Temporal Relationships Between Anxiety, Mood and Mental Imagery in Patients With Bipolar Disorder: A Network Analysis.

Clinical psychology & psychotherapy·2024
Same author

Reexamining the effects of speed-accuracy instructions with a diffusion-model-based analysis.

Journal of experimental psychology. Learning, memory, and cognition·2023
Same journal

Planned missingness in intensive longitudinal studies: Extensions and comparisons of multiform designs.

Behavior research methods·2026
Same journal

A validity-guided workflow for robust large language model research in psychology.

Behavior research methods·2026
Same journal

Are 7-point Likert scales preferable to 5-point scales in language research?

Behavior research methods·2026
Same journal

Generative psychometrics via AI-GENIE: Automatic item generation and validation with network-integrated evaluation.

Behavior research methods·2026
Same journal

Exploring psychological tradeoffs: Developing and demonstrating an R Shiny app for Pareto optimization.

Behavior research methods·2026
Same journal

The performance of Bayesian fit measures in detecting misspecified multilevel structural equation modeling.

Behavior research methods·2026
See all related articles

Related Experiment Video

Updated: Nov 9, 2025

Author Spotlight: A Novel Setup to Conduct Naturalistic Laboratory Experiments with Real Human Actors in Scenarios
07:43

Author Spotlight: A Novel Setup to Conduct Naturalistic Laboratory Experiments with Real Human Actors in Scenarios

Published on: August 4, 2023

2.4K

Do data from mechanical Turk subjects replicate accuracy, response time, and diffusion modeling results?

Roger Ratcliff1, Andrew T Hendrickson2

  • 1The Ohio State University, 1835 Neil Avenue, Columbus, OH, 43210, USA. ratcliff.22@osu.edu.

Behavior Research Methods
|April 7, 2021
PubMed
Summary
This summary is machine-generated.

Online data collection quality was assessed using cognitive tasks via Amazon Mechanical Turk (AMT). While lexical decision tasks showed good data quality, numerosity tasks had many participants with unreliable response times, impacting data analysis.

Keywords:
Across-session variabilityDiffusion decision modelMechanical Turk dataResponse time and accuracy

More Related Videos

An Open-Source, Fully Customizable 5-Choice Serial Reaction Time Task Toolbox for Automated Behavioral Training of Rodents
09:39

An Open-Source, Fully Customizable 5-Choice Serial Reaction Time Task Toolbox for Automated Behavioral Training of Rodents

Published on: January 19, 2022

4.5K
Irrelevant Stimuli and Action Control: Analyzing the Influence of Ignored Stimuli via the Distractor-Response Binding Paradigm
12:12

Irrelevant Stimuli and Action Control: Analyzing the Influence of Ignored Stimuli via the Distractor-Response Binding Paradigm

Published on: May 14, 2014

10.8K

Related Experiment Videos

Last Updated: Nov 9, 2025

Author Spotlight: A Novel Setup to Conduct Naturalistic Laboratory Experiments with Real Human Actors in Scenarios
07:43

Author Spotlight: A Novel Setup to Conduct Naturalistic Laboratory Experiments with Real Human Actors in Scenarios

Published on: August 4, 2023

2.4K
An Open-Source, Fully Customizable 5-Choice Serial Reaction Time Task Toolbox for Automated Behavioral Training of Rodents
09:39

An Open-Source, Fully Customizable 5-Choice Serial Reaction Time Task Toolbox for Automated Behavioral Training of Rodents

Published on: January 19, 2022

4.5K
Irrelevant Stimuli and Action Control: Analyzing the Influence of Ignored Stimuli via the Distractor-Response Binding Paradigm
12:12

Irrelevant Stimuli and Action Control: Analyzing the Influence of Ignored Stimuli via the Distractor-Response Binding Paradigm

Published on: May 14, 2014

10.8K

Area of Science:

  • Cognitive Psychology
  • Psychological Research Methods
  • Human-Computer Interaction

Background:

  • Online data collection surged due to the COVID-19 pandemic.
  • Ensuring the quality and reliability of data from online platforms is crucial for research validity.

Purpose of the Study:

  • To evaluate the data quality of cognitive tasks administered online using Amazon Mechanical Turk (AMT).
  • To replicate established cognitive paradigms and assess participant behavior and data reliability.

Main Methods:

  • Replication of lexical decision, item recognition, and numerosity discrimination tasks.
  • Recruitment of participants via Amazon Mechanical Turk (AMT).
  • Inclusion of IQ and math computation tests; analysis of response times (RTs) and diffusion model parameters.

Main Results:

  • Lexical decision and item recognition tasks yielded relatively well-behaved data.
  • Numerosity discrimination tasks showed a high rate of fast guesses and unstable RTs in nearly half of participants.
  • Diffusion model parameters and correlations with IQ/age were consistent with prior studies, even after outlier removal.

Conclusions:

  • Online data collection via AMT can yield reliable data for some cognitive tasks, but requires careful screening for others like numerosity discrimination.
  • Methods for data quality control, outlier detection, and participant screening are essential for online research.
  • Removing fast outliers from participants with unstable RTs did not alter the main correlational findings.