DiffAct++: Diffusion Action Segmentation.

Area of Science:

• Computer Vision
• Artificial Intelligence
• Machine Learning

Background:

• Precise temporal action segmentation is crucial for understanding long-form videos.
• Existing methods often rely on complex multi-stage, iterative refinement models.

Purpose of the Study:

• To develop a novel framework for temporal action segmentation using a denoising diffusion model.
• To improve the accuracy and efficiency of action segmentation in videos.

Main Methods:

• A denoising diffusion model framework was adapted for iterative action prediction.
• A cohesive masking strategy was proposed to capture action characteristics: position prior, boundary ambiguity, and relational dependency.
• Consistency gradient guidance was introduced to enhance conditional information during inference.

Main Results:

• The proposed method achieved state-of-the-art or comparable performance on four benchmark datasets (GTEA, 50Salads, Breakfast, Assembly101).
• The generative approach demonstrated significant potential for action segmentation tasks.

Conclusions:

• Generative models, specifically denoising diffusion models, offer a promising alternative for temporal action segmentation.
• The proposed cohesive masking and consistency gradient guidance techniques effectively improve action segmentation performance.