Predicting eye fixations with higher-level visual features.

Area of Science:

• Computational neuroscience
• Computer vision
• Human visual perception

Background:

• Two hypotheses, saliency maps and object maps, explain visual system mechanisms guiding eye fixations during free viewing of natural images.
• Computational studies often define saliency based on low-level features, while psychophysical studies suggest high-level objects are more predictive of fixations.
• The relative importance of low-level saliency versus high-level object features for predicting eye movements remains debated, with limited understanding of intermediate features.

Purpose of the Study:

• To construct and evaluate computational models based on mid-level and object-level visual features for predicting eye fixations.
• To compare the predictive performance of these higher-level feature models against established low-level feature models.
• To investigate the combined performance of mid-level and object-level models and the impact of incorporating low-level features.

Main Methods:

• Development of two computational models: one utilizing mid-level features and another using object-level features.
• Quantitative performance evaluation of the developed models against state-of-the-art low-level feature models.
• Testing across multiple benchmark natural image fixation datasets and a video fixation dataset using standard evaluation metrics.

Main Results:

• On natural image datasets, the mid-level feature model significantly outperformed state-of-the-art low-level models, while the object-level model performed less effectively.
• On a video fixation dataset, both mid-level and object-level models surpassed low-level models, with object-level features showing better performance on most metrics.
• Combining mid-level and object-level models yielded superior performance; incorporating low-level features provided negligible additional benefit across datasets.

Conclusions:

• Higher-level visual features, particularly mid-level ones, demonstrate greater efficacy than low-level features for predicting eye fixations in natural image free viewing.
• Object-level features also show promise, especially in dynamic (video) contexts, suggesting a nuanced role depending on visual content.
• Future research should focus on leveraging these higher-level representations for more accurate computational models of visual attention.