Learning Computational Models of Video Memorability from fMRI Brain Imaging

Area of Science:

• Neuroscience
• Computer Science
• Cognitive Science

Background:

• Human memory for visual media varies significantly.
• Predicting video memorability is challenging due to complex cognitive processes.

Purpose of the Study:

• To develop a computational framework for predicting video clip memorability.
• To integrate low-level audiovisual features with brain activity decoding using functional magnetic resonance imaging (fMRI).

Main Methods:

• A user study established a ground truth database for video memorability.
• Functional magnetic resonance imaging (fMRI) data were collected while subjects viewed video clips.
• A computational model was trained using joint subspace learning to correlate audiovisual features with fMRI-derived brain activity.

Main Results:

• The framework effectively integrates audiovisual features and brain activity patterns.
• The learned model can predict video memorability without requiring fMRI scans.
• Evaluations on public datasets confirmed the framework's effectiveness.

Conclusions:

• The proposed computational framework offers an efficient method for predicting video memorability.
• This approach leverages brain activity insights to enhance content analysis.
• The model has potential applications in media analysis and content creation.