Region-based Activity Recognition Using Conditional GAN.

Area of Science:

• Computer Vision
• Machine Learning
• Artificial Intelligence

Background:

• Current activity recognition methods often treat classifiers as black boxes, directly using video frames without explicit spatial localization.
• This can lead to feature extraction that is influenced by irrelevant background information, potentially reducing recognition accuracy.

Purpose of the Study:

• To develop an improved activity recognition method that incorporates spatial information of the activity.
• To enhance feature learning by focusing on activity-relevant areas within video frames.
• To achieve performance comparable to state-of-the-art systems while providing localization of the recognized activity.

Main Methods:

• A conditional generative adversarial network (cGAN) is employed to generate an activity mask, highlighting the location of the activity in each video frame.
• The generated activity mask is combined with the original image data (appended to color channels).
• This enhanced input is then fed into a VGG-LSTM network for the final activity recognition task.

Main Results:

• The proposed method achieves activity prediction accuracy comparable to existing state-of-the-art systems.
• The system successfully outlines the location of the activity simultaneously with recognition.
• Experiments on Olympic sports and trauma resuscitation datasets demonstrate the effectiveness of the mask-guided feature learning.

Conclusions:

• Incorporating activity location masks significantly improves the learning of representative features for activity recognition.
• The cGAN-generated masks effectively guide the VGG-LSTM network to focus on pertinent spatial information.
• This method offers a more interpretable and potentially more robust approach to video-based activity recognition.