Event stream learning using spatio-temporal event surface.

Area of Science:

• Computer Vision
• Neuromorphic Engineering
• Machine Learning

Background:

• Event cameras capture light intensity changes asynchronously, creating event streams.
• Efficiently encoding and learning spatiotemporal information from these streams is a significant challenge.
• Existing methods often struggle to capture both spatial and temporal correlations effectively.

Purpose of the Study:

• To propose a novel event descriptor for encoding spatio-temporal features in event streams.
• To develop a stable and efficient multi-spike learning algorithm for spiking neural networks (SNNs) to classify these features.
• To improve the classification performance of event-based vision systems.

Main Methods:

• Introduction of the spatio-temporal event surface (STES) descriptor to capture spatial and temporal event correlations.
• Development of a local-search based multi-spike learning algorithm with gradient clipping for SNNs.
• Comparison with existing feature descriptors and learning methods on event stream classification tasks.

Main Results:

• The STES descriptor accurately characterizes spatio-temporal features by considering both spatial and temporal correlations.
• The proposed multi-spike learning algorithm ensures efficient and stable learning, avoiding global search and gradient explosion.
• Experimental results show superior classification performance, particularly for event streams with complex dynamics.

Conclusions:

• The proposed STES descriptor and local-search based multi-spike learning algorithm offer a significant advancement in event-based vision.
• This approach effectively addresses the challenges of encoding and learning spatio-temporal information from event streams.
• The method demonstrates high potential for applications requiring accurate and efficient processing of dynamic visual data.