Optimizing BCPNN Learning Rule for Memory Access.

Area of Science:

• Computational neuroscience
• Artificial intelligence
• High-performance computing

Background:

• Simulating large-scale biologically plausible spiking neural networks, such as the Bayesian Confidence Propagation Neural Network (BCPNN), typically demands high-performance supercomputers with accelerators.
• The von Neumann architecture, prevalent in these systems, separates storage and computation, making memory access a significant bottleneck, even with customized hardware like ASICs.
• Existing BCPNN simulation methods often involve dual-access patterns for matrices, leading to expensive data access from DRAM.

Purpose of the Study:

• To propose an optimization technique for BCPNN simulation that enhances memory access efficiency.
• To reduce the computational cost associated with dual-access patterns in neural network simulations.
• To validate the proposed method's effectiveness and minimal error introduction.

Main Methods:

• Introduced a post-synaptic history buffer and an approximation function to eliminate the column update in matrix access.
• Organized BCPNN synaptic traces and weights to avoid dual-access patterns, favoring memory access friendliness.
• Conducted error analysis through theoretical derivations and experimental validation on a GPU platform.

Main Results:

• Reduced storage requirements by 33% and global memory access demand by over 27% compared to baseline strategies.
• Decreased the DRAM access rate by more than 5% and halved the latency of updating synaptic traces.
• Demonstrated superior performance compared to other similar optimization techniques in the literature.

Conclusions:

• The proposed optimization technique significantly improves the efficiency of BCPNN simulations by addressing memory access bottlenecks.
• The method introduces negligible errors, making it a viable approach for large-scale neural network simulations.
• The optimization strategy is applicable to other artificial neural network models utilizing a Hebbian learning rule.