Deep Learning-Based Action Detection in Untrimmed Videos: A Survey

Area of Science:

• Computer Vision
• Artificial Intelligence
• Machine Learning

Background:

• Human behavior understanding is crucial for video analysis and real-world applications.
• Current action recognition methods excel in trimmed videos, but real-world videos are often lengthy and untrimmed with sparse action segments.
• Temporal activity detection in untrimmed videos aims to precisely locate action boundaries and classify action categories.

Purpose of the Study:

• To provide a comprehensive overview of deep learning-based algorithms for temporal action detection in untrimmed videos.
• To review advances in spatio-temporal action detection and online action detection.
• To discuss benchmark datasets, evaluation metrics, and future research directions.

Main Methods:

• Review of deep learning algorithms for temporal action detection under various supervision settings (fully-supervised, weakly-supervised, unsupervised, self-supervised, semi-supervised).
• Examination of spatio-temporal action detection methods localizing actions in both time and space.
• Analysis of online action detection techniques for live video streams.

Main Results:

• The study synthesizes current deep learning approaches for temporal action detection in untrimmed videos.
• It highlights progress in spatio-temporal and online action detection.
• Performance comparisons across benchmark datasets and state-of-the-art methods are presented.

Conclusions:

• Deep learning offers powerful solutions for temporal action detection in untrimmed videos across diverse supervision levels.
• Advances in spatio-temporal and online detection are expanding the capabilities for real-world video analysis.
• Further research is needed to address challenges and explore new applications.