A Two-Stream Method for Human Action Recognition Using Facial Action Cues.

Area of Science:

• Computer Vision
• Artificial Intelligence
• Biomedical Engineering

Background:

• Human action recognition (HAR) is crucial for surveillance and healthcare.
• Existing HAR methods struggle with occlusion, often missing vital action cues.
• Facial information remains visible and informative even when the body is occluded.

Purpose of the Study:

• To introduce Face in Action (FIA), a novel two-stream method for robust action recognition under occlusion.
• To leverage facial action cues for improved accuracy in challenging real-world scenarios.
• To enhance the effectiveness of HAR in surveillance and healthcare monitoring.

Main Methods:

• Developed a two-stream approach: an RGB stream and a facial landmark stream.
• Utilized a fine-spatio-multitemporal (FSM) 3D convolution for detailed facial movement analysis.
• Employed a normalized temporal attention (NTA) module within an NTA-GCN block for landmark sequence processing.

Main Results:

• FIA demonstrated significant performance improvements over existing methods in highly occluded scenarios.
• The FSM module captured intricate local facial movements effectively.
• The NTA-GCN block enhanced the recognition of key facial frames and overall accuracy.

Conclusions:

• FIA offers a robust solution for human action recognition, particularly when facing significant occlusion.
• The method's reliance on facial cues makes it suitable for practical applications in surveillance and healthcare.
• FIA advances the field of computer vision by providing a more resilient HAR system.