TN-ZSTAD: Transferable Network for Zero-Shot Temporal Activity Detection.

Area of Science:

• Computer Vision
• Artificial Intelligence
• Machine Learning

Background:

• Temporal activity detection is crucial for video analysis and surveillance.
• Deep learning methods excel but require extensive annotated data, limiting real-world application.
• Challenges include unseen activity classes and costly data annotation.

Purpose of the Study:

• Propose a novel task: zero-shot temporal activity detection (ZSTAD).
• Develop an end-to-end deep transferable network (TN-ZSTAD) for ZSTAD.
• Enable detection of activities not encountered during training.

Main Methods:

• Designed the TN-ZSTAD network with an activity graph transformer.
• Predicts activity instances directly, avoiding pre-generated proposals.
• Utilizes label embeddings for common semantics and an innovative loss function.

Main Results:

• TN-ZSTAD demonstrates promising performance in detecting unseen activities.
• Evaluated on THUMOS'14, Charades, and ActivityNet datasets.
• The approach effectively bridges the gap between seen and unseen activity classes.

Conclusions:

• ZSTAD is a viable solution for real-world video analysis challenges.
• The TN-ZSTAD network offers an effective architecture for transferable learning in activity detection.
• This work advances the capability of AI systems to understand dynamic visual content.