Related Experiment Video
Updated: Jul 15, 2025

A Psychophysics Paradigm for the Collection and Analysis of Similarity Judgments
Published on: March 1, 2022
The impact of ChatGPT on human data collection: A case study involving typicality norming data
1Methodology and Statistics Unit, Institute of Psychology, Leiden University, Wassenaarseweg 52, 2333, AK Leiden, The Netherlands. t.d.p.heyman@fsw.leidenuniv.nl.
Abstract:
Tools like ChatGPT, which allow people to unlock the potential of large language models (LLMs), have taken the world by storm. ChatGPT's ability to produce written output of remarkable quality has inspired, or forced, academics to consider its consequences for both research and education. In particular, the question of what constitutes authorship, and how to evaluate (scientific) contributions has received a lot of attention. However, its impact on (online) human data collection has mostly flown under the radar. The current paper examines how ChatGPT can be (mis)used in the context of generating norming data. We found that ChatGPT is able to produce sensible output, resembling that of human participants, for a typicality rating task. Moreover, the test-retest reliability of ChatGPT's ratings was similar to that of human participants tested 1 day apart. We discuss the relevance of these findings in the context of (online) human data collection, focusing both on opportunities (e.g., (risk-)free pilot data) and challenges (e.g., data fabrication).
More Related Videos
07:43Author Spotlight: A Novel Setup to Conduct Naturalistic Laboratory Experiments with Real Human Actors in Scenarios
Published on: August 4, 2023
09:40Measuring Neural and Behavioral Activity During Ongoing Computerized Social Interactions: An Examination of Event-Related Brain Potentials
Published on: November 15, 2014
Related Concept Videos
Naturalistic Observations
Stereotype Content Model
Detection of Gross Error: The Q Test
Typical Model Studies
Statistical Inference Techniques in Hypothesis Testing: Parametric Versus Nonparametric Data
Parametric statistics, as the name suggests, assumes that data follow a specific distribution, often a normal distribution. This assumption enables robust hypothesis testing and estimation. Parametric methods, like the Student's t-test or Goodness-of-fit test, are frequently employed in biostatistics due to their robustness. For instance,...
Statistical Methods to Analyze Parametric Data: Student t-Test and Goodness-of-Fit Test
The Student's t-test is a statistical test that examines if there is a statistically significant difference between the means of two groups. This test is instrumental when dealing with...