Video Scene Detection Using Transformer Encoding Linker Network (TELNet).

Area of Science:

• Computer Vision
• Artificial Intelligence
• Machine Learning

Background:

• Scene boundary detection is vital for video analysis tasks like summarization.
• Existing methods often require prior knowledge of video structure.
• Accurate scene segmentation aids in organizing and understanding video content.

Purpose of the Study:

• To introduce a novel deep learning model, the Transformer Encoding Linker Network (TELNet), for unsupervised video scene boundary detection.
• To develop an efficient method for identifying scene transitions without relying on pre-defined video structures.
• To enhance the performance and scalability of automatic scene detection in videos.

Main Methods:

• Utilized a Transformer Encoding Linker Network (TELNet) architecture.
• Employed a rolling window approach to process video shots sequentially.
• Extracted shot features using a fine-tuned 3D Convolutional Neural Network (CNN) and encoded them with a transformer encoder.
• Established links between consecutive shot features using a linker component to identify discontinuities.

Main Results:

• TELNet achieved performance comparable to state-of-the-art models in standard video scene detection benchmarks.
• Demonstrated significantly improved results (F-score) in cross-dataset evaluations, indicating strong generalization.
• Exhibited linear computational complexity with respect to the number of video shots, ensuring efficiency for long videos.

Conclusions:

• TELNet offers an effective and efficient solution for automatic video scene boundary detection.
• The model's unsupervised nature and robust performance across datasets highlight its practical applicability.
• TELNet's linear scalability makes it suitable for processing large-scale video archives and real-time applications.