Defining Image Memorability Using the Visual Memory Schema

Area of Science:

• Cognitive Science
• Computer Vision
• Psychology

Background:

• Image memorability is traditionally observer-dependent, but recent work treats it as an intrinsic property.
• Existing models for image memorability prediction have limitations.
• Understanding shared human perception of image memorability is crucial for improving predictive models.

Purpose of the Study:

• To enhance the understanding and prediction of image memorability.
• To introduce and operationalize the concept of Visual Memory Schema (VMS).
• To investigate the relationship between VMS, human annotations, eye fixations, and saliency.

Main Methods:

• Human observers identified memorable image regions during an episodic memory test.
• Statistical assessment of Visual Memory Schema (VMS) consistency across observers.
• Analysis of VMS associations with eye fixations and saliency maps.
• Adaptation of deep learning architectures for memorable region reconstruction and prediction, including transfer learning.

Main Results:

• Consistent patterns in Visual Memory Schemas (VMS) were observed across observers for both correctly and incorrectly recognized images.
• Associations between VMS, eye fixations, and saliency were analyzed.
• Deep learning models demonstrated potential for reconstructing and predicting memorable image regions.

Conclusions:

• The Visual Memory Schema (VMS) offers a novel framework for understanding shared human perception in image memorability.
• Cumulative human annotations, operationalized through VMS, improve image memorability prediction.
• Deep learning approaches show promise in leveraging VMS for predicting memorable image regions.