Eyetracking for two-person tasks with manipulation of a virtual world

Area of Science:

• Human-Computer Interaction
• Cognitive Science
• Neuroscience

Background:

• Traditional eyetracking is limited to single users and static content.
• Studying joint attention and action in real-time interactive scenarios is challenging.
• Existing systems lack the capability to integrate multiple users' visual and motor data during dynamic tasks.

Purpose of the Study:

• To develop and present a novel software system for synchronized dual-user eyetracking.
• To enable the study of visual attention during coordinated joint actions.
• To facilitate the analysis of multimodal communication in collaborative tasks.

Main Methods:

• Linking two eyetracking systems in parallel with an on-screen task interface.
• Automatic tracking of dynamic on-screen objects by mapping eye movements to screen regions.
• Cross-projection of eyetrack and mouse data between participants' workspaces using spatial relationship (SR) technology.
• Recording eyetrack and on-screen events in a unified format for multimodal analysis.

Main Results:

• The system successfully synchronizes eyetracking data from two users during a cooperative task.
• It allows for the automatic tracking of moving objects relevant to the joint action.
• The cross-projection feature enables direct visualization of each user's focus within the shared workspace.
• The unified data format supports detailed analysis of multimodal interactions.

Conclusions:

• The developed software provides a new platform for researching joint action and joint attention.
• It allows for unprecedented insights into spontaneous multimodal communication during collaborative activities.
• The open-source system facilitates further research in human-computer interaction and cognitive science.