Deep Action Parsing in Videos With Large-Scale Synthesized Data

Area of Science:

• Computer Vision
• Deep Learning
• Action Recognition

Background:

• Action parsing in complex videos is a challenging computer vision problem.
• Existing methods struggle with simultaneous localization, classification, and attribute learning.

Purpose of the Study:

• To propose a generic 3D convolutional neural network for effective deep action parsing.
• To jointly optimize action localization, classification, and attribute learning.
• To simultaneously describe where, what, and how actions are performed in videos.

Main Methods:

• Developed DAP3D-Net, a 3D convolutional neural network using a multi-task learning approach.
• Trained the network on a new synthetic dataset (NASA) with 200,000 action clips and 33 attributes.
• Evaluated the model on custom Human Action Understanding and public THUMOS datasets.

Main Results:

• DAP3D-Net accurately localizes, categorizes, and describes multiple actions in realistic videos.
• Joint optimization of tasks improved overall action parsing performance.
• The NASA dataset provides a rich resource for action understanding research.

Conclusions:

• The proposed DAP3D-Net effectively addresses the challenge of deep action parsing in complex scenes.
• Multi-task learning with appearance-motion data enhances action understanding.
• The developed approach offers a comprehensive solution for video-based action analysis.