Accurate and Efficient Event-Based Semantic Segmentation Using Adaptive Spiking Encoder-Decoder Network.

Area of Science:

• Computer Vision
• Artificial Intelligence
• Neuromorphic Engineering

Background:

• Spiking Neural Networks (SNNs) offer low-power, event-driven computation suitable for dynamic signals from event-based sensors.
• Current SNNs face training and architectural challenges, limiting performance in dense prediction tasks compared to Artificial Neural Networks (ANNs).

Purpose of the Study:

• To develop an efficient Spiking Encoder-Decoder Network (SpikingEDN) for large-scale event-based semantic segmentation (EbSS).
• To improve SNNs' learning efficiency, accuracy, sparsity, and robustness in processing dynamic event streams.

Main Methods:

• Developed an efficient Spiking Encoder-Decoder Network (SpikingEDN).
• Incorporated an adaptive threshold mechanism to enhance learning from dynamic event streams.
• Introduced a dual-path spiking spatially adaptive modulation (SSAM) module to improve sparse event and multimodal input representation.

Main Results:

• SpikingEDN achieved a mean intersection over union (MIoU) of 72.57% on the DDD17 dataset.
• SpikingEDN achieved a mean intersection over union (MIoU) of 58.32% on the DSEC-Semantic dataset.
• Demonstrated competitive results against state-of-the-art ANNs with significantly reduced computational resources.

Conclusions:

• The developed SpikingEDN shows strong potential for event-based semantic segmentation tasks.
• SNNs offer a promising, computationally efficient alternative for event-based vision applications.
• This research highlights the untapped capabilities of SNNs in processing dynamic visual data.