Helping or watching it happen: how participants respond to robot failures in a turn-taking game

Robot failures in Human-Robot Interaction (HRI), though often stemming from technical limitations, can have severe effects on the interactional dynamics between humans and robots. Prior empirical research has led to conflicting findings on how such failures influence user perceptions and the overall success of the interaction. In this study, we investigate how human participants respond to robot failures on a moment-to-moment basis, with a particular focus on how social roles, responsibilities, and agency are negotiated as these episodes unfold. We examine how responses and helping behaviors are instantiated, and which factors facilitate or hinder recovery strategies. We focus on kinematic failures, such as interruptions in motion, unsuccessful grasping, or dropping objects, that occurred during Tic-Tac-Toe games between human participants (n = 17) and the humanoid robot Epi. Our analysis combines multimodal conversation analysis (MCA) and thick description, drawing on our interdisciplinary backgrounds in cognitive science and feminist Science and Technology Studies (STS). We present selected interactional sequences that illustrate a range of participant responses, including physical repair and scaffolding, interpretive support, emotional care, sustained monitoring, and dynamic negotiation of agency. These observations demonstrate how humans co-construct interactional continuity and robot competence through distributed, multimodal, and affective forms of help. They also reveal how agency is dynamically reconfigured, and how roles and responsibilities are distributed across human and robotic actors. We show how the burden of repair often falls to the human participant and conclude by reflecting on the setting and methods used, specifically in regards to the role of the robot as a research tool.