Variable Temporal Length Training for Action Recognition CNNs.

Area of Science:

• Computer Vision
• Deep Learning
• Artificial Intelligence

Background:

• Current deep learning models, particularly for computer vision, exhibit limited flexibility regarding input shape, often requiring fixed dimensions for optimal performance.
• Video analysis tasks face challenges due to the inherent variability in video lengths (number of frames), necessitating frame sampling techniques that can degrade feature quality and hinder adaptability.
• Standard training methods can damage features in longer videos and prevent models from flexibly adapting to variable lengths for on-demand inference.

Purpose of the Study:

• To propose a novel training paradigm, Variable Length Training (VLT), for 3D Convolutional Neural Networks (3D-CNNs).
• To enable 3D-CNNs to effectively process videos with variable temporal lengths without performance degradation.
• To enhance the flexibility and adaptability of deep learning models for video-related tasks.

Main Methods:

• Introduced Variable Length Training (VLT) for 3D-CNNs, incorporating three additional training operations: sampling twice, temporal packing, and subvideo-independent 3D convolution.
• Integrated these efficient operations into existing 3D-CNN architectures.
• Implemented a consistency loss to regularize the representation space, further enhancing model robustness.

Main Results:

• The proposed VLT method allows trained models to process videos of varying temporal lengths during inference without any architectural modifications.
• Experiments on popular action recognition datasets demonstrated superior performance compared to conventional training paradigms.
• The method showed improved results over other state-of-the-art training approaches for variable length video processing.

Conclusions:

• Variable Length Training (VLT) offers a simple yet effective solution for deep learning models to handle variable length video inputs.
• The VLT paradigm enhances model flexibility, adaptability, and performance in video analysis tasks, particularly action recognition.
• This approach overcomes the limitations of fixed-length input requirements in current deep learning models for video processing.