TODO-Net: Temporally Observed Domain Contrastive Network for 3-D Early Action Prediction.

Area of Science:

• Computer Vision
• Machine Learning
• Artificial Intelligence

Background:

• Early action prediction is challenging due to domain gaps in temporal data.
• Existing methods often overlook discrepancies between low- and highly-observed temporal domains, leading to performance degradation.

Purpose of the Study:

• To propose a novel Temporally Observed Domain Contrastive Network (TODO-Net) for 3-D early action prediction.
• To explicitly mine discrimination information from hard-to-classify action samples in low-observed temporal domains.

Main Methods:

• TODO-Net leverages relationships between low-observed and highly-observed sequences of the same action category.
• A temporal domain conditioned supervised contrastive (TD-conditioned SupCon) learning scheme is introduced to minimize intra-category domain gaps and maximize inter-category separation.

Main Results:

• TODO-Net effectively mines discrimination information from hard action samples.
• The proposed TD-conditioned SupCon scheme mitigates domain gaps within action categories.

Conclusions:

• TODO-Net significantly boosts recognition performance for actions with fewer observed frames.
• The approach demonstrates efficacy on public 3-D skeleton-based activity datasets.