Jove
Visualize
Contact Us
JoVE
x logofacebook logolinkedin logoyoutube logo
ABOUT JoVE
OverviewLeadershipBlogJoVE Help Center
AUTHORS
Publishing ProcessEditorial BoardScope & PoliciesPeer ReviewFAQSubmit
LIBRARIANS
TestimonialsSubscriptionsAccessResourcesLibrary Advisory BoardFAQ
RESEARCH
JoVE JournalMethods CollectionsJoVE Encyclopedia of ExperimentsArchive
EDUCATION
JoVE CoreJoVE BusinessJoVE Science EducationJoVE Lab ManualFaculty Resource CenterFaculty Site
Terms & Conditions of Use
Privacy Policy
Policies

Related Concept Videos

You might also read

Related Articles

Articles linked to this work by shared authors, journal, and citation graph.

Sort by
Same author

Kinematic Parameter Estimation Using Workspace Manifold Mapping.

IEEE transactions on bio-medical engineering·2025
Same author

Gaze-Based Shared Autonomy Framework With Real-Time Action Primitive Recognition for Robot Manipulators.

IEEE transactions on neural systems and rehabilitation engineering : a publication of the IEEE Engineering in Medicine and Biology Society·2023
Same author

Elastohydrodynamic friction of robotic and human fingers on soft micropatterned substrates.

Nature materials·2021
Same author

Toward Shared Autonomy Control Schemes for Human-Robot Systems: Action Primitive Recognition Using Eye Gaze Features.

Frontiers in neurorobotics·2020
Same author

Medical robotics-Regulatory, ethical, and legal considerations for increasing levels of autonomy.

Science robotics·2020
Same author

Perception of Tactile Directionality via Artificial Fingerpad Deformation and Convolutional Neural Networks.

IEEE transactions on haptics·2020

Related Experiment Video

Updated: Nov 19, 2025

Estimation of Contact Regions Between Hands and Objects During Human Multi-Digit Grasping
09:41

Estimation of Contact Regions Between Hands and Objects During Human Multi-Digit Grasping

Published on: April 21, 2023

2.0K

Exploiting Three-Dimensional Gaze Tracking for Action Recognition During Bimanual Manipulation to Enhance Human-Robot

Alireza Haji Fathaliyan1, Xiaoyu Wang1, Veronica J Santos1

  • 1Biomechatronics Laboratory, Mechanical and Aerospace Engineering, University of California, Los Angeles, Los Angeles, CA, United States.

Frontiers in Robotics and AI
|January 27, 2021
PubMed
Summary

This study introduces a novel method using 3D gaze behavior for human action recognition, achieving 96.4% accuracy. This advancement enables robots to better understand and support human actions in collaborative tasks.

Keywords:
action recognitionbimanual manipulationeye trackinggaze fixationgaze object sequencegaze saliency maphuman–robot collaborationinstrumental activity of daily living

More Related Videos

Efficiently Recording the Eye-Hand Coordination to Incoordination Spectrum
07:30

Efficiently Recording the Eye-Hand Coordination to Incoordination Spectrum

Published on: March 21, 2019

8.1K
Design and Use of an Apparatus for Presenting Graspable Objects in 3D Workspace
09:11

Design and Use of an Apparatus for Presenting Graspable Objects in 3D Workspace

Published on: August 8, 2019

5.9K

Related Experiment Videos

Last Updated: Nov 19, 2025

Estimation of Contact Regions Between Hands and Objects During Human Multi-Digit Grasping
09:41

Estimation of Contact Regions Between Hands and Objects During Human Multi-Digit Grasping

Published on: April 21, 2023

2.0K
Efficiently Recording the Eye-Hand Coordination to Incoordination Spectrum
07:30

Efficiently Recording the Eye-Hand Coordination to Incoordination Spectrum

Published on: March 21, 2019

8.1K
Design and Use of an Apparatus for Presenting Graspable Objects in 3D Workspace
09:11

Design and Use of an Apparatus for Presenting Graspable Objects in 3D Workspace

Published on: August 8, 2019

5.9K

Area of Science:

  • Human-Robot Interaction
  • Computer Vision
  • Machine Learning

Background:

  • Intuitive human-robot collaboration requires robots to recognize human actions and infer intent.
  • Traditional action recognition uses 2D video analysis, limiting depth perception.
  • Three-dimensional (3D) gaze behavior offers a richer data source for understanding human actions.

Purpose of the Study:

  • To identify useful features from 3D gaze behavior for human action recognition.
  • To develop and evaluate a machine learning algorithm using these features.
  • To enhance human-robot collaboration through improved action recognition.

Main Methods:

  • Reconstructed 3D gaze vectors and object interactions using motion capture and eye tracking.
  • Analyzed gaze fixation duration and saccade size during a daily living task (preparing a drink).
  • Developed a "gaze object sequence" feature and used dynamic time warping for analysis.

Main Results:

  • Identified that certain actions (pouring, stirring) require more visual attention.
  • Achieved 96.4% accuracy, 89.5% precision, and 89.2% recall in action recognition using the gaze object sequence.
  • Demonstrated the feasibility of a simple action recognition algorithm with high performance.

Conclusions:

  • 3D gaze object sequences are a promising feature for robust human action recognition.
  • This approach can significantly improve human-robot collaboration in various settings.
  • Future work can enhance real-time implementation with advanced machine learning classifiers.