Video Transformers: A Survey.

Area of Science:

• Computer Vision
• Artificial Intelligence
• Deep Learning

Background:

• Transformer models demonstrate success in handling long-range dependencies, crucial for video data.
• Existing Transformers lack inductive biases and exhibit quadratic scaling with input length, limiting their video modeling capabilities.
• Current surveys on Transformers for vision do not offer an in-depth analysis of video-specific architectural designs.

Purpose of the Study:

• To provide a comprehensive analysis of Transformer-based approaches for video modeling.
• To investigate trends and contributions in leveraging Transformers for video understanding.
• To address the limitations of standard Transformers in handling the high dimensionality and temporal dynamics of video data.

Main Methods:

• Analysis of input-level processing strategies for video data in Transformer architectures.
• Study of architectural modifications to enhance efficiency, reduce redundancy, and incorporate inductive biases for video.
• Exploration of training regimes and self-supervised learning strategies tailored for video Transformers.
• Performance comparison on action classification benchmarks against 3D Convolutional Networks (ConvNets).

Main Results:

• Video Transformers demonstrate superior performance in action classification compared to 3D ConvNets.
• Optimized Transformer designs achieve better efficiency, often with reduced computational complexity.
• Effective self-supervised learning strategies significantly enhance video Transformer performance.
• Architectural innovations address the quadratic scaling issue and re-introduce beneficial inductive biases.

Conclusions:

• Transformer models, with tailored designs, are highly effective for video modeling tasks.
• Video-specific Transformer architectures offer a promising direction for advancing video understanding.
• These models present a computationally efficient and high-performing alternative to traditional video analysis methods.