RECOGNIZING BEHAVIOR IN HAND-EYE COORDINATION PATTERNS

Area of Science:

• Robotics and Human-Computer Interaction
• Artificial Intelligence and Machine Learning
• Cognitive Science and Behavioral Modeling

Background:

• Accurate modeling of human behavior is crucial for designing intuitive robots and human-computer interfaces (HCIs).
• Existing constructive models often lack the detailed, moment-to-moment coordination of hand, head, and eye movements observed in complex human tasks.
• Understanding and replicating these intricate behavioral dynamics is a significant challenge in HCI and robotics.

Purpose of the Study:

• To develop a computational model capable of capturing the detailed structure of human behavior during complex tasks.
• To demonstrate a method for programming a dynamic Bayes network (DBN) using observational data of human task performance.
• To enable real-time recognition of human actions and task instances using the developed DBN.

Main Methods:

• Collected detailed kinematic and observational data from human subjects performing a complex task (e.g., sandwich making).
• Utilized this data to program a dynamic Bayes network (DBN), a probabilistic graphical model suitable for sequential data.
• Implemented the DBN for real-time inference and recognition of task performance instances.

Main Results:

• The programmed DBN successfully captured the nuanced, moment-to-moment coordination of hand, head, and eye gaze.
• The DBN demonstrated the ability to recognize new instances of task performance in real time.
• Specific complex activities, such as sandwich making, were accurately recognized by the DBN.

Conclusions:

• Dynamic Bayes networks provide a powerful framework for modeling and recognizing complex human behaviors.
• This approach advances the development of more sophisticated and responsive robots and human-computer interfaces.
• Real-time recognition of human actions using DBNs opens new possibilities for intelligent systems and assistive technologies.