Real-time signal processing enabled by fused networks on a memristor-based system on a chip

The von Neumann bottleneck has led to a substantial rise in energy consumption of computing hardware and memory systems, particularly for data-intensive tasks like signal processing. Memristor-based in-memory computing offers an efficient alternative by performing computations within analog memory. Here, we demonstrate real-time signal processing using a fused network that combines the real-time discrete Fourier transform (DFT) and convolutional neural network (CNN) on a memristor-based analog system on a chip (SoC). A 128-by-128 memristor crossbar array performs the DFT on audio signals with a peak signal-to-noise ratio of 33.49 dB, while the following CNN classifies the resulting spectrograms with 94.72% accuracy on the AudioMNIST dataset. In addition, convolution-based edge detection is applied to real-time video frames. The SoC offers substantial energy efficiency improvement over traditional digital systems in signal processing tasks. This work highlights the potential of memristor-based SoCs for efficient real-time signal processing.