Exploring the role of gaze behavior and object detection in scene understanding

Area of Science:

• Computer Vision
• Human-Computer Interaction
• Cognitive Science

Background:

• Gaze behavior during scene viewing offers insights into human attention and scene understanding.
• Previous research has explored links between visual attention and scene content, but integrating gaze data with computational models is ongoing.

Purpose of the Study:

• To investigate the information encoded in human gaze behavior during free-viewing of scenes.
• To explore the relationship between eye fixations, scene descriptions, and object detection predictions.
• To develop prototype human-in-the-loop applications for gaze-enabled computer vision tasks.

Main Methods:

• Utilized two popular computer vision image datasets for experiments.
• Analyzed the relationship between human eye fixations, scene descriptions, and automatic object category detection.
• Combined human gaze data with outputs from current visual recognition methods.

Main Results:

• Demonstrated that gaze patterns contain significant information about scene content and viewer's focus.
• Established correlations between fixation points, descriptive language used by viewers, and object detection results.
• Successfully built prototype applications leveraging human gaze for object detection and scene annotation.

Conclusions:

• Human gaze behavior is a rich source of information for understanding scenes and guiding computational models.
• Integrating human visual attention data with computer vision enhances object detection and scene understanding.
• Gaze-enabled systems offer promising avenues for human-computer interaction in visual analysis tasks.