3D and 4D Image Fusion: Coping with Differences in Spectroscopic Modes among Hyperspectral Images

Image fusion is often oriented to solve differences in spatial scale and orientation among different spectroscopic platforms. However, an additional problem arises when the nature of the spectroscopic information differs in dimensionality as well. Indeed, most imaging systems, e.g., Raman, IR, MS, etc., allow acquisition of 3D images, with a linear spectrum per pixel, but new platforms have emerged, such as the recent excitation-emission fluorescence imaging platforms that provide 4D images, with a 2D spectral landscape per pixel. A proper 3D/4D image fusion needs to take into account the difference in the dimension of the spectral information and in the underlying models of both measurements (bilinear for 3D images and trilinear for 4D images). This work solves this image fusion problem through a new dedicated variant of the multivariate curve resolution-alternating least squares (MCR-ALS) algorithm for multiset analysis based on the incorporation of a hybrid bilinear/trilinear model that can handle the image fused structure preserving the natural behavior of the 3D and 4D imaging techniques coupled. The example is illustrated on the fusion of real 3D Raman and 4D fluorescence images recorded on cross sections of rice leaf samples.