Combinatorial optimization by weight annealing in memristive hopfield networks

Area of Science:

• Hardware accelerators
• Computational neuroscience
• Materials science

Background:

• Optimization problems are increasingly important, driving demand for efficient hardware solutions.
• Hopfield neural networks offer a promising avenue for combinatorial optimization, especially with mixed-signal implementations using non-volatile memory.
• Annealing techniques are crucial for finding optimal solutions in these networks.

Purpose of the Study:

• To propose and validate a novel "weight annealing" approach for Hopfield neural networks.
• To enhance the convergence to global minima in optimization problems.
• To demonstrate the practical application of this method using emerging memory devices.

Main Methods:

• Developed a "weight annealing" strategy, starting with zero synaptic weights and gradually introducing them.
• Conducted extensive numerical simulations on representative combinatorial problems.
• Experimentally validated the approach using mixed-signal circuits with TiO2 memristive crossbars and eFlash memory arrays.

Main Results:

• The "weight annealing" method consistently yielded better average solutions compared to chaotic or stochastic annealing in Hopfield neural network solvers.
• Successfully solved a 13-node graph partitioning problem and a 7-node maximum-weight independent set problem experimentally.
• Demonstrated the efficacy of mixed-signal hardware accelerators for optimization tasks.

Conclusions:

• The proposed "weight annealing" technique is an effective strategy for improving Hopfield neural network performance in solving combinatorial optimization problems.
• Mixed-signal accelerators based on emerging non-volatile memory devices are viable for implementing advanced annealing techniques.
• This work paves the way for more efficient hardware-based optimization solutions.