Asymmetric Multi-Task Learning for Interpretable Gaze-Driven Grasping Action Forecasting.

Area of Science:

• Robotics
• Human-Computer Interaction
• Artificial Intelligence

Background:

• Predicting human grasping intention is crucial for assistive robotics.
• Human attention, particularly gaze fixations, is a strong indicator of intention.
• Existing methods for action anticipation and grasping prediction have limitations.

Purpose of the Study:

• To develop a multi-task system for simultaneously predicting human attention and grasping actions.
• To improve assistive robotics for individuals with motor and cognitive disabilities.
• To leverage human visual attention patterns for enhanced action anticipation.

Main Methods:

• A multi-task deep learning system analyzing first-person video and eye-tracking data (gaze fixations).
• Modeling visual attention as a competitive process between discrete states representing gaze movement patterns.
• Employing an asymmetric multi-task learning approach with a constrained multi-task loss, where attention prediction aids grasping action anticipation.

Main Results:

• The proposed model achieved state-of-the-art performance on three egocentric action anticipation datasets.
• Achieved average precision of 0.569 (GITW) and 0.524 (Sharon).
• Demonstrated high accuracy (89.2%) and success rate (51.7%) on the Invisible dataset.

Conclusions:

• The multi-task system effectively predicts human attention and grasping intentions.
• The asymmetric learning approach with constrained loss enhances action prediction accuracy.
• This research offers significant advancements for human-robot interaction in assistive applications.