Propensity to trust in Large Language Models

Area of Science:

• Artificial Intelligence
• Human-Computer Interaction
• Cognitive Science

Background:

• Trust is crucial for human-AI collaboration, yet the trust tendencies of large language models (LLMs) remain largely unexplored.
• Propensity to Trust (PTT) is a stable individual difference in humans; its existence and nature in LLMs are unknown.
• Understanding LLM trust is vital for safe and effective deployment in collaborative environments.

Purpose of the Study:

• To investigate whether large language models (LLMs) exhibit a Propensity to Trust (PTT).
• To differentiate between stable baseline trust tendencies and context-dependent trust adjustments in LLMs.
• To identify factors influencing trust decisions in LLMs during collaborative tasks.

Main Methods:

• Administered a psychological self-report scale adapted for humans to nineteen LLMs to assess PTT.
• Employed a linguistic simulation framework to elicit trust-related decisions in LLMs across various contexts.
• Conducted ablation studies to examine the role of memory mechanisms in trust calibration.

Main Results:

• Questionnaire-based PTT measures were uniformly high across LLMs, likely due to social-alignment objectives.
• Linguistic simulations revealed significant, systematic differences in LLM trust behaviors, indicating varying PTT.
• More capable models (e.g., GPT-4o-mini) adjusted trust based on trustworthiness cues, while less capable models (e.g., Llama-2-7B) showed stable, evidence-insensitive delegation.
• Task-specific memory mechanisms improved LLMs' ability to integrate trustworthiness cues and calibrate delegation.

Conclusions:

• LLM trust behavior is a complex interplay between baseline delegation tendencies and the capacity to integrate contextual trustworthiness cues.
• Questionnaire methods are insufficient for distinguishing stable PTT from context-sensitive trust adjustments in LLMs.
• Behavioral simulations are essential for accurately assessing and understanding LLM trust dynamics and calibration.