A Video Summarization Model Based on Deep Reinforcement Learning with Long-Term Dependency.

Area of Science:

• Computer Science
• Artificial Intelligence
• Machine Learning

Background:

• Deep summarization models using Gated Recursive Unit (GRU) and Long Short-Term Memory (LSTM) have advanced video summarization.
• Existing models struggle with capturing long-term dependencies in extended videos.

Purpose of the Study:

• To propose a deep summarization network that overcomes the limitations of GRU and LSTM in handling long videos.
• To introduce an unsupervised framework for improved video summarization without requiring labels or user interaction.

Main Methods:

• Developed a deep summarization network incorporating an unsupervised auxiliary summarization loss module with LSTM and a swish activation function.
• Implemented a reward function considering summary consistency, diversity, and representativeness.
• Designed a lightweight model deployable on mobile devices.

Main Results:

• The unsupervised approach achieved superior video summaries compared to existing methods on benchmark datasets.
• Demonstrated a significant increase in F scores: 6.3% on SumMe and 2.2% on TVSum datasets compared to the DR-DSN model.
• Validated the model's effectiveness in capturing long-term dependencies.

Conclusions:

• The proposed unsupervised deep summarization network effectively addresses the challenge of long-term dependencies in video summarization.
• The lightweight and unsupervised nature of the model offers practical advantages for mobile deployment and server efficiency.
• The approach significantly improves summarization quality, outperforming previous state-of-the-art methods.