Neural sampling machine with stochastic synapse allows brain-like learning and inference.

Area of Science:

• Neuromorphic Engineering
• Artificial Intelligence
• Probabilistic Machine Learning

Background:

• Mission-critical applications demand continual online learning from noisy data and real-time decision-making with confidence.
• Brain-inspired probabilistic neural networks offer inherent uncertainty handling and adaptive learning but face hardware implementation challenges.
• Compact, low-power hardware for probabilistic models is crucial for advancing AI capabilities.

Purpose of the Study:

• To introduce a novel hardware fabric for implementing a new class of stochastic neural networks, Neural Sampling Machines (NSMs).
• To exploit synaptic stochasticity for approximate Bayesian inference in a hardware-efficient manner.
• To demonstrate the potential of NSMs for continual online learning and uncertainty estimation in AI.

Main Methods:

• Development of a novel hardware fabric implementing Neural Sampling Machines (NSMs).
• Experimental demonstration of an in silico hybrid stochastic synapse using a ferroelectric field-effect transistor (FeFET) and a stochastic selector.
• Network-level simulations to evaluate NSM performance in autonomous weight normalization, Bayesian inference, and image classification.

Main Results:

• The stochastic synapse emulates multiplicative synaptic noise essential for NSM operation.
• Stochastic NSMs demonstrated autonomous weight normalization for continual online learning.
• Achieved 98.25% accuracy on MNIST image classification and successfully estimated prediction uncertainty for rotated digits.

Conclusions:

• The proposed hardware fabric enables efficient implementation of stochastic neural networks for AI.
• Stochastic NSMs offer significant advantages for continual learning and uncertainty quantification in real-time applications.
• This probabilistic hardware platform enhances AI's learning and inference capabilities, inspired by neuroscience.