Deep Liquid State Machines With Neural Plasticity for Video Activity Recognition

Area of Science:

• Artificial Intelligence
• Computational Neuroscience
• Machine Learning

Background:

• Edge devices face limitations in processing resource-intensive AI algorithms due to size, weight, and power constraints.
• Traditional reservoir computing offers computational frugality but sacrifices performance compared to state-of-the-art methods.
• A need exists for efficient yet powerful algorithms for real-world applications like video activity recognition on edge devices.

Purpose of the Study:

• To introduce deep-LSM networks as a novel deep spiking neural network architecture.
• To evaluate the performance of deep-LSM networks for video activity recognition on resource-constrained edge devices.
• To demonstrate that deep-LSM networks can overcome the limitations of traditional reservoir computing while maintaining low computational costs.

Main Methods:

• Developed deep-LSM networks, a deep spiking neural network featuring randomly connected and unsupervised layers.
• Incorporated an attention-modulated readout layer for processing dynamic information captured over multiple time-scales.
• Evaluated the network on the DogCentric video activity recognition benchmark.

Main Results:

• Achieved an average accuracy of 84.78% on the DogCentric video activity recognition task, outperforming state-of-the-art methods.
• Demonstrated significant memory savings of up to 91.13%.
• Showcased a reduction in synaptic operations by up to 91.55% compared to similar recurrent neural network models.

Conclusions:

• Deep-LSM networks provide an effective compromise between reservoir computing and fully supervised networks for edge AI.
• The proposed architecture successfully captures dynamic information across multiple time-scales with high accuracy and efficiency.
• Deep-LSM networks represent a promising approach for enabling advanced AI capabilities on intelligent edge devices.