Parameter-Efficient Adaptation of Large Vision-Language Models for Video Memorability Prediction

Area of Science:

• Artificial Intelligence
• Computer Vision
• Multimedia Analysis

Background:

• Accurate video memorability modeling is crucial for efficient media retrieval, classification, and generation.
• Strong correlation exists between video visual semantics and memorability, necessitating advanced visual comprehension.
• Large Vision-Language Models (LVLMs) excel at high-level semantic understanding due to extensive multimodal pre-training.

Purpose of the Study:

• To leverage LVLMs for video memorability prediction.
• To explore efficient adaptation techniques for LVLMs in memorability modeling.
• To investigate the impact of LoRA hyperparameters on memorability prediction performance.

Main Methods:

• Fine-tuning the Qwen-VL model using the Quantized Low-Rank Adaptation (QLoRA) technique.
• Utilizing memorability-related data from the Memento10k dataset for adaptation.
• Transforming Qwen-VL into a memorability score regressor.
• Optimizing LoRA hyperparameters (rank and alpha) via 5-Fold Cross-Validation.

Main Results:

• Achieved a state-of-the-art Spearman Rank Correlation Coefficient (SRCC) of 0.744 on the Memento10k dataset.
• Demonstrated the effectiveness of QLoRA for adapting LVLMs to memorability prediction.
• Identified optimal LoRA hyperparameters for improved performance.

Conclusions:

• This work significantly advances video memorability modeling through LVLMs and efficient adaptation.
• The proposed methodology offers a robust approach for predicting video memorability.
• High-level semantic understanding is key to accurate video memorability prediction.