Related Experiment Video
Updated: May 24, 2025
08:48
Optical Recording of Suprathreshold Neural Activity with Single-cell and Single-spike Resolution
Published on: September 5, 2012
11.8K
Self-Supervised High-Order Information Bottleneck Learning of Spiking Neural Network for Robust Event-Based Optical
Summary
This study introduces SeLHIB, a novel self-supervised learning algorithm for event cameras, enhancing optical flow estimation in noisy conditions. SeLHIB improves robustness and energy efficiency, outperforming existing methods.
Area of Science:
- Computer Vision
- Neuromorphic Engineering
- Machine Learning
Background:
- Event cameras excel in high-speed, high-dynamic-range visual perception.
- Deep learning for event-based optical flow estimation needs better temporal feature capture.
- Spiking Neural Networks (SNNs) offer potential for efficient sequential data processing but struggle with generalization and robustness.
Purpose of the Study:
- To introduce SeLHIB, a self-supervised learning algorithm for robust event-based optical flow estimation.
- To leverage information bottleneck theory within SNNs for improved spatiotemporal feature extraction.
- To enhance generalization and robustness in noisy visual scenes.
Main Methods:
- Developed a novel spike-based self-supervised learning algorithm, SeLHIB, utilizing information bottleneck theory.
- Employed nonlinear and high-order mutual information for enhanced information extraction and redundancy reduction.
- Trained and evaluated the algorithm using event-based camera inputs under various noise conditions.
Main Results:
- SeLHIB demonstrated significantly enhanced generalization and robustness in optical flow estimation across different noise levels.
- Achieved substantial energy savings: 90.44% reduction compared to Analog Neural Network (ANN) counterparts and 45.70% compared to Spiking Neural Network (SNN) counterparts.
- Outperformed ANN implementations with comparable sizes and architectures, showing lower AEE (MVSEC) by 33.78% and lower RSAT (ECD) by 5.96%, and RSAT (HQF) by 6.21%.
Conclusions:
- SeLHIB represents the first self-supervised information bottleneck learning strategy for SNNs in event-based vision.
- The proposed method effectively addresses the limitations of current SNNs in generalization and robustness for optical flow tasks.
- SeLHIB offers a promising direction for energy-efficient and robust visual perception systems using event cameras.
