BIDL: a brain-inspired deep learning framework for spatiotemporal processing.

Area of Science:

• Neuroscience
• Machine Learning
• Computer Vision

Background:

• Deep spiking neural networks (DSNNs) emulate biological brains for spatiotemporal perception (STP), particularly with dynamic vision sensor (DVS) data.
• Existing frameworks lack generalization for diverse spatiotemporal data like videos and 3D imaging.

Purpose of the Study:

• Introduce a unified training platform, Brain-Inspired Deep Learning (BIDL), for generalized spatiotemporal processing (STP).
• Investigate lightweight STP using brain-inspired neural dynamics.
• Provide a research framework for neuroscience and machine learning exploration.

Main Methods:

• Developed the BIDL framework for constructing deep neural networks.
• Integrated neural dynamics for temporal processing and artificial neural network layers for spatial accuracy.
• Implemented optimizations for neuromorphic chips and GPUs (iteration representation, state-aware computational graph, built-in neural functions).

Main Results:

• Demonstrated BIDL's efficiency across diverse data: video, DVS, 3D medical imaging, and natural language processing.
• Showcased BIDL's capability for generalized spatiotemporal processing.
• Validated BIDL as a user-friendly and efficient DSNN builder.

Conclusions:

• BIDL offers a unified and efficient solution for generalized spatiotemporal processing using brain-inspired deep learning.
• The framework facilitates exploration of neural models and co-learning, advancing bio-inspired research.
• BIDL supports lightweight STP applications, adaptable for neuromorphic hardware and GPUs.