Memristor-based multilayer neural networks with online gradient descent training

Area of Science:

• Neuroscience
• Computer Engineering
• Materials Science

Background:

• Multilayer neural networks (MNNs) require extensive synaptic weight updates for learning.
• Implementing these updates using traditional CMOS technology is area- and power-intensive.
• Existing hardware implementations face challenges in compact and efficient multiply-accumulate operations.

Purpose of the Study:

• To propose a novel memristor-based circuit for simultaneous synaptic weight updates in MNNs.
• To demonstrate a hardware-efficient method for implementing incremental outer product learning.
• To reduce the area and power consumption of MNN hardware accelerators.

Main Methods:

• Utilized memristor arrays for performing simultaneous incremental outer product operations.
• Developed a synaptic circuit comprising one memristor and two CMOS transistors.
• Leveraged the memristor's conductivity change proportional to voltage pulse magnitude and duration.

Main Results:

• The proposed memristor-based synaptic circuit is significantly more compact than CMOS-only alternatives.
• Expected area and static power consumption are reduced by 92%–98% compared to previous hardware.
• Demonstrated the circuit's utility and robustness on standard supervised learning tasks.

Conclusions:

• Memristor-based circuits offer a compact and power-efficient solution for MNN hardware.
• The proposed method enables scalable implementation of MNNs trainable by online gradient descent.
• This approach paves the way for more efficient artificial intelligence hardware.