Layer ensemble averaging for fault tolerance in memristive neural networks

Area of Science:

• Computer Science
• Materials Science
• Electrical Engineering

Background:

• Conventional computing faces memory bottlenecks, limiting artificial neural network (ANN) advancements.
• In-memory computing with memristor devices shows potential but suffers from hardware non-idealities.
• Developing fault-tolerant architectures is crucial for reliable memristive neural networks.

Purpose of the Study:

• To propose and validate a hardware-oriented fault tolerance scheme called layer ensemble averaging.
• To improve the inference performance of non-ideal memristive neural networks.
• To demonstrate the scheme's effectiveness on both simulated and hardware platforms.

Main Methods:

• Implementing layer ensemble averaging for fault tolerance in memristive neural networks.
• Conducting simulations for an image classification task with programmed pre-trained solutions.
• Performing hardware experiments on a continual learning problem using a 20,000-device prototyping platform.

Main Results:

• Significant performance gains observed in both image classification and continual learning tasks.
• For image classification with 20% faults, accuracy improved from 40% to 89.6%.
• For continual learning, accuracy increased from 55% to 71% with minimal performance overhead.

Conclusions:

• Layer ensemble averaging effectively mitigates hardware non-idealities in memristive neural networks.
• The proposed scheme offers substantial performance improvements over prior methods at similar redundancy levels.
• The fault tolerance approach is broadly applicable to various non-volatile device-based accelerators.