A Trained Humanoid Robot can Perform Human-Like Crossmodal Social Attention and Conflict Resolution

Area of Science:

• Robotics
• Cognitive Science
• Human-Computer Interaction

Background:

• Effective human-robot interaction necessitates robots processing multiple social cues in real-world environments.
• Information incongruency across sensory modalities presents a significant challenge for robot social cognition.
• Existing systems often struggle with integrating and resolving conflicting social signals.

Purpose of the Study:

• To develop a neurorobotic approach for crossmodal conflict resolution in social attention.
• To enable robots to exhibit human-like social attention responses in complex scenarios.
• To investigate the impact of audio-visual congruency on social attention in both humans and robots.

Main Methods:

• Conducted a behavioral experiment with 37 participants in a simulated round-table meeting with masked avatars.
• Manipulated spatial congruency between avatar eye gaze (central) and sound location (peripheral).
• Trained a saliency prediction model on social cues to detect, predict, and selectively attend to audio-visual information for robot application on the iCub platform.

Main Results:

• Human participants demonstrated better performance in congruent audio-visual conditions compared to incongruent ones.
• Dynamic gaze shifts from a central avatar effectively triggered crossmodal social attention responses.
• The robot, equipped with the trained model, replicated human-like attention responses, despite overall lower performance than humans.

Conclusions:

• Crossmodal conflict resolution is crucial for robots to achieve human-like social attention.
• The developed saliency prediction model shows promise in enabling robots to process and respond to social cues.
• This research advances the field of social robotics by providing a framework for more sophisticated human-robot interaction.