Event-Based Machine Vision for Edge AI Computing

Area of Science:

• Computer Vision
• Artificial Intelligence of Things (AIoT)
• Edge Computing

Background:

• Event-based sensors, like Dynamic Vision Sensors (DVS), offer sparse, motion-centric data ideal for low-bandwidth, always-on perception.
• Resource-constrained edge devices require efficient algorithms for real-time AI tasks.

Purpose of the Study:

• To develop an event-based machine vision framework for smart-home AIoT applications.
• To enable efficient human/object detection, 2D human pose estimation, and hand posture recognition using event data.

Main Methods:

• Developed timestamp-based, polarity-agnostic recency encoding to preserve motion structure and reduce background noise.
• Optimized task-specific neural networks through architectural reduction and mixed-bit quantization for sparse event images.
• Evaluated performance on human detection, pose estimation, and hand posture recognition tasks.

Main Results:

• Raw DVS data stream size reduced by ~30x compared to conventional CMOS video.
• Human detection computation reduced by over 11x (5.8 G to 81 M FLOPs) with a runtime speed-up from 172 ms to 15 ms.
• Pose estimation model size reduced from 127 MB to 19 MB, with inference time decreasing from 70 ms to 6 ms, maintaining high accuracy.
• Hand posture recognition achieved 99.19% recall with 14.31 ms latency.

Conclusions:

• Event-based sensing combined with lightweight inference is a practical solution for privacy-friendly, real-time perception on edge devices.
• The proposed framework demonstrates significant improvements in data efficiency and computational performance for smart-home AIoT.