Real-time classification and sensor fusion with a spiking deep belief network

Area of Science:

• Computational Neuroscience
• Machine Learning
• Neuromorphic Engineering

Background:

• Deep Belief Networks (DBNs) excel at classification but are computationally intensive on serial systems.
• Generative properties of DBNs aid performance understanding and sensor fusion.
• Efficient hardware implementation of DBNs is crucial for real-time applications.

Purpose of the Study:

• To propose a method for mapping offline-trained DBNs to efficient, event-driven spiking neural networks (SNNs).
• To enable hardware implementation of DBNs using Integrate-and-Fire neurons and the Siegert approximation.
• To demonstrate real-time visual classification and sensor fusion using neuromorphic sensors.

Main Methods:

• Utilized the Siegert approximation for Integrate-and-Fire neurons to approximate DBNs.
• Developed an event-driven SNN architecture suitable for hardware deployment.
• Implemented and tested a 3-layer SNN for MNIST digit classification using a Dynamic Vision Sensor (DVS) and an AER-EAR silicon cochlea.

Main Results:

• Achieved real-time handwritten digit recognition (MNIST) using event-based neuromorphic sensors.
• Demonstrated robust recognition against noise, scaling, translation, and rotation with <1% performance degradation.
• Real-time recognition averaged 5.8 ms post-stimulus onset; sensory fusion improved accuracy with ambiguous inputs.

Conclusions:

• The proposed method enables efficient, hardware-implementable DBNs using event-driven SNNs.
• The system offers high-speed, robust, and low-power recognition capabilities for real-world applications.
• Multi-sensory fusion enhances classification accuracy, particularly in challenging conditions.