Real-Time Biologically Inspired Action Recognition from Key Poses Using a Neuromorphic Architecture.

Area of Science:

• Robotics
• Computer Vision
• Artificial Intelligence

Background:

• Intelligent agents require autonomous functions like navigation and human interaction.
• Recognizing human actions is crucial for seamless human-robot collaboration.
• Existing methods often lack efficiency and real-time capabilities for mobile systems.

Purpose of the Study:

• To develop a biologically inspired visual architecture for human action recognition.
• To implement an energy-efficient system for real-time action classification.
• To evaluate the performance and generalization of the proposed method.

Main Methods:

• Utilized a biologically inspired visual architecture processing form and motion information separately.
• Employed an event-based scheme identifying characteristic human pose configurations (key poses).
• Trained a deep convolutional neural network using the energy-efficient deep neuromorphic networks (Eedn) framework and mapped it to the IBM Neurosynaptic System.

Main Results:

• Achieved real-time action recognition at approximately 1,000 frames per second with low energy consumption (70 mW).
• Demonstrated performance on par with state-of-the-art key pose-based methods.
• Key pose representations showed higher confidence in class assignments and promising cross-dataset generalization.

Conclusions:

• The proposed approach offers an efficient and effective solution for real-time human action recognition in robotic systems.
• The energy-efficient design is particularly suitable for mobile robotic applications.
• Unsupervised key pose selection and deep neuromorphic networks provide a robust framework for action recognition.