Related Experiment Video
Updated: Jul 17, 2025

Characterization of the Sense of Agency over the Actions of Neural-machine Interface-operated Prostheses
Published on: January 7, 2019
Exploring the effect of automation failure on the human's trustworthiness in human-agent teamwork
Carolina Centeio Jorge1, Nikki H Bouman1, Catholijn M Jonker1,2
1Interactive Intelligence, Intelligent Systems Department, Delft University of Technology, Delft, Netherlands.
Abstract:
Introduction: Collaboration in teams composed of both humans and automation has an interdependent nature, which demands calibrated trust among all the team members. For building suitable autonomous teammates, we need to study how trust and trustworthiness function in such teams. In particular, automation occasionally fails to do its job, which leads to a decrease in a human's trust. Research has found interesting effects of such a reduction of trust on the human's trustworthiness, i.e., human characteristics that make them more or less reliable. This paper investigates how automation failure in a human-automation collaborative scenario affects the human's trust in the automation, as well as a human's trustworthiness towards the automation. Methods: We present a 2 × 2 mixed design experiment in which the participants perform a simulated task in a 2D grid-world, collaborating with an automation in a "moving-out" scenario. During the experiment, we measure the participants' trustworthiness, trust, and liking regarding the automation, both subjectively and objectively. Results: Our results show that automation failure negatively affects the human's trustworthiness, as well as their trust in and liking of the automation. Discussion: Learning the effects of automation failure in trust and trustworthiness can contribute to a better understanding of the nature and dynamics of trust in these teams and improving human-automation teamwork.
Related Concept Videos
Stereotype Content Model
Distribution Reliability and Automation
Fundamental Attribution Error
Obedience
Milgram's Obedience to Authority
Social Loafing

