Spatially Arranged Sparse Recurrent Neural Networks for Energy Efficient Associative Memory

Area of Science:

• Artificial Intelligence
• Neuroscience
• Computer Engineering

Background:

• Hardware neural networks, including neuromorphic hardware, are rapidly advancing.
• Operating large-scale neural networks on low-power devices is challenging due to high communication costs from numerous interconnections.

Purpose of the Study:

• To develop learning algorithms for energy-efficient information processing in hardware neural networks.
• To address communication costs by designing sparse recurrent neural networks with improved cost-performance ratios for associative memory.

Main Methods:

• Investigated two approaches for creating spatially arranged sparse recurrent neural networks.
• First approach: utilized sparse modular network structures inspired by biological brains and iterative learning.
• Second approach: formulated an optimization problem to maximize network sparsity under pattern embedding constraints.

Main Results:

• Incorporating long-range connections in modular networks improved cost-performance ratio.
• Optimized networks demonstrated a tradeoff between interconnection cost and computational performance.
• Optimized networks achieved superior cost-performance ratios compared to modular networks.

Conclusions:

• The developed approaches are effective in identifying sparse and cost-efficient neural network connectivity.
• These methods can enhance energy efficiency in hardware neural networks.
• The optimization approach shows promise for applications like binary pattern processing and grayscale image restoration.