The ChatGPT source cue shapes answer acceptance in a reasoning task
Related Experiment Video
Updated: Nov 9, 2026

Using Eye Movements Recorded in the Visual World Paradigm to Explore the Online Processing of Spoken Language
Published on: October 13, 2018
Rosa Angela Fabio1, Elisabetta Rantuccio2, Francesca Bruno2
1Department of Biomedical, Morphological and Functional Imaging Sciences, University of Messina, Messina, Italy.
Introduction:
As generative AI systems increasingly provide answers in everyday and academic contexts, understanding how source attribution influences evaluative judgment has become critical.
Methods:
In a within-subjects design, 103 participants evaluated 30 pre-generated reflective-correct, intuitive-incorrect, and arithmetic control-correct answers labeled ChatGPT or computer. Assignment of two parallel batteries to source labels and trial presentation order were randomized, and displays were visually matched. An independent unlabeled pretest with 35 participants supported battery comparability.
Results:
Participant-level analysis showed greater overall acceptance under ChatGPT (68.0% versus 63.4%, p = 0.002). A trial-level logistic mixed model with crossed participant and parallel-problem-pair random intercepts showed a source-by-category interaction (p < 0.001): acceptance was higher under ChatGPT for arithmetic controls (OR = 6.08, 95% CI [2.70, 13.69]), while source contrasts within the two CRT categories were not significant. Judgment accuracy is reported descriptively. Signal-detection analyses of CRT answers detected no significant source difference in sensitivity (p = 0.930) or criterion (p = 0.082). Evaluation times were longer under ChatGPT in the raw-time analysis (p = 0.034), but the source difference was not significant after log transformation (p = 0.180). Exploratory correlations detected no significant associations between source-related differences and trust, AI attitudes, or critical-thinking disposition.
Discussion:
These findings distinguish overall source-related endorsement, particularly for arithmetic controls, from successful discrimination between correct and incorrect CRT answers.
Related Concept Videos
Hypothesis: Accept or Fail to Reject?
There are two ways to indicate that the null hypothesis is not rejected. 'Accept' the null hypothesis and 'fail to...
Inductive Reasoning
Deductive Reasoning
Reasoning
Inductive reasoning involves deriving generalizations from specific observations. This type of reasoning helps form beliefs about the world. For example,...
The Anchoring-and-Adjustment Heuristic
The Representativeness Heuristic

