Overtrust in AI Recommendations About Whether or Not to Kill: Evidence from Two Human-Robot Interaction Studies

Area of Science:

• Human-Computer Interaction
• Artificial Intelligence Ethics
• Cognitive Psychology

Background:

• Trust in artificial agents is crucial for human-AI collaboration.
• Understanding factors influencing trust in AI decision-making is vital, especially in high-stakes scenarios.

Purpose of the Study:

• To investigate determinants of trust in artificial agents' recommendations for kill decisions under uncertainty.
• To examine the impact of agent embodiment and anthropomorphism on trust and decision-making.

Main Methods:

• A novel visual challenge paradigm simulating threat identification (enemy combatants vs. civilians).
• Comparison of trust in embodied vs. screen-mediated anthropomorphic robots.
• Manipulation of virtual robot anthropomorphism levels.

Main Results:

• No significant effect of embodiment on trust was observed.
• Increased anthropomorphism of virtual agents led to modestly greater trust.
• Participants frequently reversed threat identifications and kill decisions when AI disagreed, degrading performance.
• Subjective confidence tracked AI agreement, moderated by perceived agent intelligence.

Conclusions:

• Humans exhibit a strong propensity to overtrust unreliable AI in life-or-death decisions under uncertainty.
• Perceived intelligence of AI agents influences trust and decision-reversal patterns.
• AI recommendations, even when incorrect, can substantially impair human judgment in critical situations.