Computational discrimination between natural images based on gaze during mental imagery

Area of Science:

• Cognitive Neuroscience
• Computer Vision
• Human-Computer Interaction

Background:

• Humans exhibit spontaneous eye movements during image recall, mirroring spatial content.
• These imagery eye movements act as memory cues, aiding retrieval.
• The precise correlation between imagery and encoding eye movements remains under-explored.

Purpose of the Study:

• To quantify the similarity between eye movements during image encoding and recall.
• To investigate the feasibility of using imagery eye movements for computational image retrieval.
• To assess the generalizability of this eye-movement-based retrieval method to novel images.

Main Methods:

• Quantifying the similarity between eye movement data collected during image encoding and spontaneous recall.
• Developing and testing a computational image retrieval system based on these eye movement similarities.
• Evaluating the system's performance on both familiar and unseen images.

Main Results:

• A strong correlation was found between eye movements during image encoding and spontaneous recall.
• Computational image retrieval using imagery eye movements proved to be feasible.
• The proposed retrieval method demonstrated generalizability to images not encountered during training.

Conclusions:

• Imagery eye movements during spontaneous recall contain robust information about visual memory.
• Eye movement patterns during recall can be effectively leveraged for computational image retrieval.
• This approach offers a novel, memory-driven method for image search and recognition.