"Reading Pictures Instead of Looking": RGB-D Image-Based Action Recognition via Capsule Network and Kalman Filter

Area of Science:

• Computer Vision
• Machine Learning
• Deep Learning

Background:

• Convolutional Neural Networks (CNNs) exhibit over-sensitivity to rotation and scaling.
• Interpretability of CNN models is limited, especially regarding spatial relationships.
• Action recognition requires robust methods that handle variations in pose and scale.

Purpose of the Study:

• To propose a novel action recognition algorithm, Reading Pictures Instead of Looking (RPIL), that overcomes CNN limitations.
• To enhance the interpretability of action recognition models by leveraging spatial coordinates.
• To improve computational efficiency and accuracy in human action recognition.

Main Methods:

• Utilizing capsule networks to extract human body components and their attributes.
• Employing Bert for analyzing detected parts and attribute parameters.
• Integrating a Kalman filter to refine capsule predictions and mitigate misinformation.
• Pruning the network structure into a dendritic network for enhanced computational efficiency.

Main Results:

• Achieved 83.3% precision on the OAD dataset, 72.2% on ChaLearn Gesture, and 86.5% on G3D dataset.
• Demonstrated real-time operation capabilities with a frame rate exceeding 30 fps.
• Minimized dependence on random features extracted by CNNs without compromising accuracy.

Conclusions:

• The RPIL algorithm offers improved interpretability and robustness in action recognition compared to traditional CNNs.
• The integration of capsule networks and Kalman filters effectively addresses challenges in rotation and scaling.
• The proposed method achieves high accuracy and real-time performance, suitable for practical applications.