A Data-Driven Approach for Comparing Gaze Allocation Across Conditions

Area of Science:

• Cognitive Science
• Neuroscience
• Computer Vision

Background:

• Traditional gaze analysis uses subjective regions of interest (ROIs) or heatmaps, limiting objectivity and exploration.
• ROIs offer condition comparisons but reduce objectivity, while heatmaps require extensive pixel-wise analysis, hindering difference detection.
• Existing methods struggle to reveal nuanced gaze behaviors and the impact of external stimuli like sound on visual attention.

Purpose of the Study:

• To develop and validate an advanced, data-driven approach for analyzing gaze behavior using deep neural networks (DNNs).
• To objectively investigate the influence of object-specific sounds on gaze allocation and identify underlying visual attention strategies.
• To compare the efficacy of the proposed method against traditional ROI-based and heatmap analyses.

Main Methods:

• Utilized adapted versions of AlexNet (deep neural networks) to classify experimental conditions based on gaze patterns.
• Employed reverse correlation techniques to pinpoint specific areas and patterns of gaze differences between conditions.
• Tested the approach on experimental data examining the effects of congruent sound, no sound, phase-scrambled sound, and pink noise on gaze allocation.

Main Results:

• The DNNs achieved significantly above-chance accuracy in classifying tasks from individual fixations in certain scenes.
• ROI-based analysis showed significant differences in gaze allocation towards sound-associated objects in the congruent sound condition, but significance was ROI-dependent.
• Heatmap analysis revealed unclear qualitative differences, with no significant findings after correcting for pixel-wise comparisons.

Conclusions:

• The proposed data-driven method objectively demonstrates that sound significantly alters gaze allocation strategies.
• The study uncovered non-trivial, task-specific visual attention strategies where fixations shift away from salient features, not always towards the sound source.
• This advanced approach provides a more objective and exploratory method for gaze analysis, revealing insights unattainable with traditional hypothesis-driven techniques.