Detection, Inspection, Return: An Object-Based Classification and Metric of Fixations in Complex Scenes

Area of Science:

• Cognitive Science
• Computer Vision
• Neuroscience

Background:

• Traditional analyses of human gaze behavior in complex scenes, such as heatmaps and scan-paths, have limitations.
• Heatmaps lack temporal information, while scan-paths can be overly detailed and impractical for analysis.
• A new object-centric approach is needed to better capture the spatiotemporal dynamics of visual attention.

Purpose of the Study:

• To introduce and validate a novel classification system for human fixations based on object-centric temporal context.
• To test the hypothesis that Detection (D), Inspection (I), and Return (R) fixations represent distinct behavioral profiles.
• To demonstrate the utility of this classification for analyzing gaze behavior in dynamic scenes.

Main Methods:

• Reanalysis of a large dataset of human scene fixations.
• Computation of separate heatmaps for D, I, and R fixations to assess inter-observer consistency.
• Development and training of a semantic salience model to predict fixation types and analyze feature weights.
• Analysis of fixation type proportions across viewing time and trial duration.

Main Results:

• Significantly higher inter-observer consistency was found within D, I, and R fixation classes compared to between classes.
• The proportion of D, I, and R fixations varied consistently across different semantic features.
• A semantic salience model showed diverging feature weight distributions when trained to predict each fixation type independently.
• A temporal shift from Detection to Inspection and Return fixations was observed, varying with trial duration.

Conclusions:

• The D, I, R classification provides a computationally simple yet powerful method for analyzing spatiotemporal aspects of scene fixations.
• This object-based approach offers an intuitive way to understand visual attention compared to traditional methods.
• The D, I, R classification is a valuable metric for gaze comparisons, especially in dynamic scenes where scan-path metrics falter.
• Future research should explore potential functional differences between Detection, Inspection, and Return fixations.