That was not what I was aiming at! Differentiating human intent and outcome in a physically dynamic throwing task

Area of Science:

• Human-Robot Interaction
• Robotics
• Computer Vision

Background:

• Recognizing human intent is crucial for effective human-robot collaboration, impacting team performance and robot perception.
• Mistakes are common in dynamic physical tasks, leading to a discrepancy between intended and observed actions.
• Existing methods for intent recognition struggle with the nuances of human error in real-time tasks.

Purpose of the Study:

• To develop a robust method for recognizing human intent in collaborative tasks, even when mistakes occur.
• To leverage facial expressions as a key indicator of a throw's outcome and the user's underlying intent.
• To improve the accuracy and efficiency of intent recognition systems in human-robot teams.

Main Methods:

• A dataset of 1227 ball throws was collected from 10 participants, with detailed observation of throw outcomes and facial reactions.
• Facial images were analyzed to predict whether a throw was a mistake and to infer the actual intent behind the action.
• A novel 1D-Convolutional Neural Network (1D-CNN) model was developed, incorporating mistake frequency priors for end-to-end outcome and intent recognition.

Main Results:

• 47% of throws were identified as mistakes, with 16% missing the target entirely.
• The proposed outcome prediction approach demonstrated a 38% performance improvement over previous two-stream architectures on front-on videos.
• The 1D-CNN model, combined with mistake priors, provided an effective end-to-end pipeline for recognizing both outcome and intent.

Conclusions:

• Facial cues are valuable predictors of task outcomes and user intent in human-robot collaboration.
• The developed 1D-CNN model offers a significant advancement in accurately recognizing intent, even in the presence of errors.
• This research contributes to more intuitive and effective human-robot teaming through enhanced intent recognition capabilities.