View-invariant Deep Architecture for Human Action Recognition using Two-stream Motion and Shape Temporal Dynamics.

Area of Science:

• Computer Vision
• Machine Learning
• Human-Computer Interaction

Background:

• Human action recognition from unknown viewpoints presents significant challenges.
• Existing methods often struggle with view variations, limiting their real-world applicability.
• Effective action recognition requires capturing both motion and appearance dynamics.

Purpose of the Study:

• To develop a deep learning framework for robust, view-invariant human action recognition.
• To integrate complementary action cues: motion and shape temporal dynamics (STD).
• To achieve state-of-the-art performance on challenging action recognition benchmarks.

Main Methods:

• A novel deep framework combining motion and STD streams.
• Motion stream: RGB Dynamic Images (RGB-DIs) via Approximate Rank Pooling (ARP), processed by InceptionV3.
• STD stream: LSTM and Bi-LSTM models on view-invariant features from Human Pose Model (HPM).
• Late fusion techniques (max, average, multiply) for final prediction.

Main Results:

• The framework demonstrates superior performance on NUCLA, UWA3D-II, and NTU RGB+D datasets.
• Significant improvements in recognition accuracy, ROC curves, and AUC compared to state-of-the-art methods.
• Effective view-invariant feature extraction using HPM and STD stream.

Conclusions:

• The proposed deep view-invariant framework effectively integrates motion and shape dynamics for human action recognition.
• The novel approach achieves significant performance gains, outperforming existing methods.
• The framework shows strong generalization capabilities across different datasets and validation schemes.