A Hybrid Neuromorphic Object Tracking and Classification Framework for Real-Time Systems.

Area of Science:

• Neuromorphic Engineering
• Computer Vision
• Deep Learning

Background:

• Deep learning inference on edge devices is computationally intensive, limiting low-power applications.
• Event-based cameras offer low power consumption and high dynamic range, suitable for embedded systems.

Purpose of the Study:

• To propose a real-time, hybrid neuromorphic framework for object tracking and classification on low-power platforms.
• To achieve energy savings and high performance using a mixed frame and event-based approach.

Main Methods:

• A hybrid framework combining frame-based region proposal with event-based processing.
• Implementation of a hardware-friendly object tracking scheme using apparent object velocity and occlusion handling.
• Classification using the TrueNorth (TN) neuromorphic processor via an energy-efficient deep network (EEDN) pipeline.
• Development of an alternative continuous-time tracker processing individual events for low-latency applications.

Main Results:

• The proposed system demonstrates effective object tracking and classification in practical surveillance scenarios using custom datasets.
• Extensive comparisons show comparable or superior performance to state-of-the-art event-based and frame-based methods.
• The neuromorphic approach is validated for real-time, embedded applications without performance sacrifice.

Conclusions:

• The hybrid neuromorphic framework enables efficient deep learning inference on resource-constrained edge devices.
• This approach effectively leverages the advantages of event-based cameras for object tracking and classification.
• The system shows promise for real-time embedded vision applications, including traffic monitoring.