UArch: A Super-Resolution Processor With Heterogeneous Triple-Core Architecture for Workloads of U-Net Networks

High-resolution medical images are of critical significance to improve disease diagnosis. Limited by the camera and power of medical devices, medical images often have very low resolution. For example, wireless capsule endoscopes, often used to diagnose diseases of the small bowel, can only capture low-resolution endoscopic images. The existing super-resolution (SR) networks perform exceptionally well in recovering high-resolution images, but they are computationally expensive and require high bandwidth, which can result in unacceptable latency and bandwidth requirements for embedded medical devices. In this paper, we propose a U-Net-based SR (USR) network structure and an SR processor named UArch. The USR-s, which is the lightweight version of USR, has an SR performance of 42.68 dB for ×2 scale SR. The USR-s has 0.3 dB higher PSNR (peak signal-to-noise ratio) than the SR algorithm, which is often used in recent SR hardware. Based on well-designed strategies, including heterogeneous triple-core architecture, fine-grained on-chip memory allocation, out-of-order execution, and sub-tensor-based processing flow, the UArch, designed for U-Net networks, can fulfill ×2, ×3, and ×4 scale SR by deploying USR-s, achieving high throughput of 60 fps and low latency of 25 ms for ×2 scale 1920 × 1080 output image SR at 156 MHz. The UArch achieves high energy efficiency which is 2264.5 GOPS/W when synthesized and evaluated under the TSMC 28 nm process and which is 199.3 GOPS/W when implemented on Xilinx ZCU111. Our SR processor is capable of reconstructing high-quality endoscopic images and is more efficient than the previous state-of-the-art SR processors.