An On-Demand Neuromorphic Vision System Enabled by a Multi-Paradigm Neuromorphic Device and Hierarchical...

There are two general approaches in guiding intelligent vision system development: one emphasizes ultra-flexibility (reconfigurability) for adapting to various scenarios, and the other emphasizes ultrahigh power efficiency tailored to specific applications. The pinnacle design is geared toward the biological vision system with concurrent high levels of on-demand intelligence, efficiency, and flexibility. However, current state-of-the-art intelligent vision systems are far behind, relying on heterogeneously integrated and limited-function single devices, alongside rigid sensing/computing architecture, thus preventing flexibility for low area and power efficiency toward dynamic and unpredictable scenarios. This work bridges the neuromorphic gap with an on-demand ultra-reconfigurable vision system, demonstrating true reconfigurability across device, cell, array and system levels. This is enabled by a multi-paradigm device array capable of seamless switching between spiking, non-spiking, neuromorphic imaging (NI), and artificial intelligence (AI) computing modes, as well as a reconfigurable circuit and architecture design. The system is capable of on-demand allocating resources between NI and AI functionalities for high-quality smart imaging and high-accuracy recognition tasks, and transitioning between spiking and non-spiking modes for frameless dynamic and frame-based static scenarios. Superior power efficiencies of up to 52.6 TOPS/W for NI-centric computing and 75.5 TOPS/W for NI/AI hybrid computing are achieved, which are up to two orders of magnitude larger than the state-of-the-art intelligent vision system.