Humans and LLMs rate deliberation as superior to intuition on complex reasoning tasks

Area of Science:

• Cognitive psychology
• Artificial intelligence
• Human-computer interaction

Background:

• Dual-process theories propose human thinking involves intuition and deliberation.
• Public perception of these reasoning types remains underexplored.
• Understanding these perceptions is crucial for human and AI recommendation trust.

Purpose of the Study:

• To investigate human preferences between intuitive and deliberative reasoning.
• To determine if these human preferences are replicated in large language models (LLMs).
• To explore the implications for trust in human and AI-generated recommendations.

Main Methods:

• 13 studies involving human participants rating reasoning quality in vignettes.
• Vignettes varied reasoning type (intuitive vs. deliberative) and past accuracy.
• LLM studies (ChatGPT 3.5 and 4) were conducted to mirror human judgment.

Main Results:

• Humans consistently rated deliberative reasoning as superior to intuition, irrespective of accuracy.
• Deliberative thinkers were perceived as smarter and more trustworthy.
• LLMs demonstrated the same preference pattern, mirroring human folk beliefs about reasoning.

Conclusions:

• Humans possess an intuitive preference for deliberation, associating it with reliability.
• LLMs capture and replicate human folk beliefs regarding reasoning quality.
• Findings have significant implications for public trust in both human and AI recommendations.