Efficient Human Vision Inspired Action Recognition Using Adaptive Spatiotemporal Sampling.

Area of Science:

• Computer Vision
• Machine Learning
• Human-Computer Interaction

Background:

• Always-on action recognition on resource-constrained wearable devices requires efficient video sampling.
• Fixed sampling strategies are context-unaware, leading to under-sampling and reduced efficiency and accuracy.
• Human visual perception offers insights into adaptive sampling mechanisms.

Purpose of the Study:

• To develop a novel adaptive spatiotemporal sampling scheme for efficient action recognition.
• To enhance computational efficiency and accuracy in video analysis for wearable devices.
• To leverage principles of human visual perception for improved sampling strategies.

Main Methods:

• Implemented an adaptive spatiotemporal sampling scheme inspired by foveal vision and pre-attentive processing.
• The system performs low-resolution global scene pre-scanning to identify salient regions.
• High-resolution feature extraction is selectively applied to these salient regions.

Main Results:

• The proposed adaptive sampling significantly speeds up inference compared to fixed strategies.
• Achieved comparable accuracy to state-of-the-art baselines with reduced computational load.
• Demonstrated effectiveness on EPIC-KITCHENS and UCF-101 datasets.

Conclusions:

• Adaptive spatiotemporal sampling is a viable approach for efficient action recognition on wearable devices.
• Mimicking human visual processing can lead to more effective computational strategies.
• The developed method offers a practical solution for real-time video analysis with limited resources.