Shot Boundary Detection with 3D Depthwise Convolutions and Visual Attention.

Area of Science:

• Computer Vision
• Machine Learning
• Video Processing

Background:

• Shot boundary detection is crucial for video analysis and scene segmentation.
• 3D convolutional networks excel at spatiotemporal feature extraction for this task.
• High computational cost of 3D convolutions hinders real-time applications.

Purpose of the Study:

• To develop a computationally efficient shot boundary detection model.
• To improve video segmentation speed for live and near-live streaming services.
• To mitigate the parameter and resource demands of 3D convolutional networks.

Main Methods:

• Utilized depthwise separable convolutions to reduce model parameters.
• Implemented a novel scheme for parameter reduction in convolutional networks.
• Incorporated visual self-attention mechanisms to enhance performance.

Main Results:

• Achieved significant reduction in model parameters compared to standard 3D convolutions.
• Demonstrated effectiveness of depthwise separable convolutions for shot boundary detection.
• Visual self-attention compensated for potential performance degradation.

Conclusions:

• The proposed method offers an efficient alternative for real-time shot boundary detection.
• This approach is suitable for enhancing user experience in streaming platforms.
• Combining depthwise separable convolutions with self-attention is a promising direction.