Multi-label zero-shot human action recognition via joint latent ranking embedding.

Area of Science:

• Computer Vision
• Machine Learning
• Artificial Intelligence

Background:

• Human action recognition is challenging, with most methods limited to single-label classification.
• Real-world videos often contain multiple actions, posing a multi-label learning problem.
• Annotating sufficient data for all possible actions is difficult, hindering supervised learning.

Purpose of the Study:

• To address the limitations of single-label and supervised approaches in human action recognition.
• To formulate real-world human action recognition as a multi-label zero-shot learning problem.
• To propose a novel framework that handles multiple actions and unseen actions.

Main Methods:

• A joint latent ranking embedding framework is proposed.
• The framework uses two component neural networks for visual and semantic embedding.
• It tackles unknown temporal action boundaries and exploits semantic relationships for zero-shot learning.

Main Results:

• The framework effectively performs multi-label zero-shot recognition by measuring relatedness scores.
• Evaluated on Breakfast and Charades datasets using a novel data split scheme.
• Experimental results demonstrate the framework's effectiveness.

Conclusions:

• The proposed joint latent ranking embedding framework advances multi-label zero-shot human action recognition.
• The approach successfully handles complex scenarios with multiple and unseen actions.
• This work provides a robust solution for recognizing human actions in real-world video streams.