Rethinking Efficient and Effective Point-Based Networks for Event Camera Classification and Regression

Area of Science:

• Computer Vision
• Neuromorphic Engineering
• Machine Learning

Background:

• Event cameras offer low latency and high dynamic range with minimal power consumption.
• Current frame-based processing of event data is computationally intensive and loses temporal details.
• Existing point-based methods struggle with spatio-temporal event streams.

Purpose of the Study:

• To develop an efficient and effective framework for processing event camera data using Point Cloud representation.
• To address the limitations of frame-based and previous point-based methods.
• To enhance the extraction of temporal information from event streams.

Main Methods:

• Proposed EventMamba, a framework utilizing Point Cloud representation for event camera data.
• Implemented a hierarchical structure with staged modules for processing temporal features.
• Redesigned the global extractor using temporal aggregation and State Space Model (SSM) based Mamba for enhanced temporal extraction.

Main Results:

• Achieved state-of-the-art (SOTA) point-based performance on six action recognition datasets.
• Outperformed all frame-based methods on Camera Pose Relocalization (CPR) and eye-tracking regression tasks.
• Demonstrated minimal computational resource consumption.

Conclusions:

• EventMamba effectively bridges the gap between event cloud and point cloud representations.
• The proposed method excels in capturing spatio-temporal information for various tasks.
• EventMamba represents a significant advancement in efficient and effective event camera data processing.