Action Recognition From a Single Coded Image.

Area of Science:

• Computer Vision
• Machine Learning
• Deep Learning

Background:

• Deep convolutional neural networks (CNNs) excel at human action recognition but require high-resolution videos and substantial resources.
• Budgetary and environmental constraints necessitate simpler, more efficient recognition models and camera systems.

Purpose of the Study:

• To introduce a deep sensing solution for direct human action recognition from coded exposure images.
• To develop a novel knowledge distillation framework for joint training of encoder and recognition models.
• To validate the feasibility of the proposed deep sensing solution with a prototype camera.

Main Methods:

• A binary CNN-based encoder network emulates coded exposure image capture.
• A subsequent 2D CNN performs action recognition on these coded images.
• A knowledge distillation framework jointly trains the encoder and action recognition model.

Main Results:

• The proposed training approach significantly improves action recognition accuracy: 6.2% on Something-v2, 2.9% on Kinetics-400, and 7.9% on UCF-101.
• A prototype coded exposure camera using LCoS was successfully built and tested.
• Prototype evaluation results align with simulation findings, confirming feasibility.

Conclusions:

• The deep sensing solution effectively recognizes human actions from coded exposure images, offering a viable alternative for resource-limited scenarios.
• The novel knowledge distillation framework enhances recognition accuracy.
• The developed prototype demonstrates the practical applicability of this approach.