Sparse improved spectral super-resolution from RGB response through tristimulus color congruency and a pixelwise...

Spectral super-resolution (SSR) from RGB response can achieve flexible applications and high resolution (both spatial and spectral) under low-cost conditions, which is beneficial in various tasks like surface coating, target recognition, and vegetation monitoring. However, most existing SSR works are based on the overall features of the image to pursue lower average errors, while such averaging effect could weaken the improvement of higher reconstruction accuracy for individual pixels, and also leads to deeper and more complex models. To address these problems, we propose a two-stage method focused on the local phase and color similarity from a pixel perspective. In the first stage, a selection strategy named Tristimulus Color Congruency (TCC) is introduced to extract the color similarity of optical images and guide the selection of pixels to enrich the training process, which is very sparse compared to the whole image (less than 1%). In the second stage, the Spectral Extension Network (SEN) with several easy-to-deploy blocks is constructed to complete an effective and efficient single-pixel spectrum prediction. Single pixel computation time of our model is compressed to the order of μs, and the method is evaluated in three public benchmarks (Munsell-1269, CAVE-real, and AeroRIT). In the hyperspectral image reconstruction task, our method improves the reconstruction performance by over 40% overall test compared to the baseline and shows consistency in different environments. Besides, as pixel-based, our method can also serve as a plug-and-play model to improve the performance on any region of any size or discrete pixels of interest. The reference code is available at https://github.com/Eahein-H/TCC-SEN.