Learning Action Distribution Flow for Open-Set Temporal Action Segmentation.

Area of Science:

• Computer Vision
• Machine Learning
• Artificial Intelligence

Background:

• Open-set temporal action segmentation aims to identify unknown frames alongside known actions.
• Current methods struggle with distinguishing unknown frames from ambiguous known frames during action transitions, leading to performance issues.

Purpose of the Study:

• To develop a novel approach for open-set temporal action segmentation.
• To effectively identify unknown frames by modeling transitions between action sequences.

Main Methods:

• Proposed the action distribution flow to model transitions and capture feature discrepancies between unknown and known frames.
• Modeled known action distributions from training data.
• Interpolated action distributions along optimal transport paths for testing videos.
• Evaluated frame likelihood against the modeled action distribution flow.

Main Results:

• Successfully identified unknown frames without additional training or prior knowledge.
• Demonstrated superior performance across all evaluation metrics on benchmark datasets (GTEA, 50Salads, Breakfast).

Conclusions:

• The action distribution flow effectively addresses the challenge of identifying unknown frames in open-set temporal action segmentation.
• The proposed method offers a robust solution for improving temporal action segmentation accuracy in the presence of unknown actions.