Braille letter reading: A benchmark for spatio-temporal pattern recognition on neuromorphic hardware.

Area of Science:

• Robotics and Neuromorphic Engineering
• Artificial Intelligence and Machine Learning
• Computational Neuroscience

Background:

• Deep learning excels at spatio-temporal pattern recognition but is computationally expensive for embedded systems.
• Real-time, energy-efficient tactile sensing is crucial for robotic applications.
• Brain-inspired computing offers potential solutions for edge-based processing.

Purpose of the Study:

• To establish a novel benchmark for spatio-temporal tactile pattern recognition at the edge using Braille letter reading.
• To investigate the efficacy of event-based encoding and spike-based computation for tactile sensing.
• To compare neuromorphic hardware (Intel Loihi) with traditional embedded GPUs (NVIDIA Jetson) for this task.

Main Methods:

• Recorded a new dataset of Braille letters using the iCub robot's tactile sensors.
• Trained and compared feedforward and recurrent Spiking Neural Networks (SNNs) using Backpropagation Through Time (BPTT).
• Deployed SNNs on the Intel Loihi neuromorphic chip and compared them against Long Short-Term Memory (LSTM) on NVIDIA Jetson GPU.

Main Results:

• Long Short-Term Memory (LSTM) achieved ~97% accuracy, outperforming recurrent SNNs by ~17% with frame-based data.
• Recurrent SNNs on Loihi with event-based inputs were ~500 times more energy-efficient than LSTMs on Jetson (~30 mW total power).
• Event-based encoding and spike-based computation present challenges and opportunities for edge AI.

Conclusions:

• Neuromorphic hardware and SNNs offer significant energy efficiency advantages for edge-based spatio-temporal pattern recognition.
• While LSTMs currently offer higher accuracy, SNNs show promise for low-power, real-time applications.
• The proposed benchmark facilitates further research into efficient edge AI solutions for tactile sensing.