An inverse Yarbus process: predicting observers' task from eye movement patterns.

Area of Science:

• Cognitive Science
• Computer Vision
• Neuroscience

Background:

• Understanding viewer intent is crucial for analyzing visual attention.
• Eye movement data offers rich insights into cognitive processes during image viewing.
• Existing methods for inferring visual tasks from eye movements have limitations.

Purpose of the Study:

• To develop a probabilistic method for inferring visual tasks from eye movement trajectories.
• To leverage hidden Markov models (HMM) for predicting fixation points based on viewer tasks.
• To apply Bayesian inference for selecting the most probable task.

Main Methods:

• Utilized hidden Markov models (HMM) with a first-order Markov process.
• Employed Bayesian inference with maximum a posteriori (MAP) probability for task selection.
• Applied a clustering technique in conjunction with the HMM approach.
• Tested the method on eye movement data from subjects viewing real scene images and answering questions.

Main Results:

• The HMM-based method demonstrated reliability in inferring visual tasks.
• The combination of HMM and clustering proved effective for analyzing complex eye movement data.
• Accurate task inference was achieved even with challenging datasets.

Conclusions:

• The developed probabilistic HMM method is a reliable tool for inferring visual tasks from eye movement data.
• This approach offers a robust framework for understanding viewer behavior in visual environments.
• Future work can explore extensions to dynamic scenes and more complex cognitive tasks.