pHSCNN: CNN-based hyperspectral recovery from a pair of RGB images

To increase the fidelity of hyperspectral recovery from RGB images, we propose a pairwise-image-based hyperspectral convolutional neural network (pHSCNN) to recover hyperspectral images from a pair of RGB images, obtained by the same color sensor with and without an optical filter in front of the imaging lens. The proposed method avoids the pitfall of requiring multiple color sensors to obtain different RGB images and achieves higher accuracy than recovery from single RGB image. Besides, pHSCNN can also optimize the optical filter to further improve the performance. To experiment on real data, we built a dual-camera hyperspectral imaging system and created a real-captured hyperspectral-RGB dataset. Experimental results demonstrate the superiority of pHSCNN with the highest accuracy of the recovered hyperspectral signature perceptually and numerically.