FUSE-Net: Multi-Scale CNN for NIR Band Prediction from RGB Using GNDVI-Guided Green Channel Enhancement.

Area of Science:

• Remote Sensing
• Computer Vision
• Plant Science

Background:

• Hyperspectral imaging (HSI) offers advanced vegetation analysis but faces limitations due to high equipment costs and complex data acquisition.
• Accessible alternatives are needed to broaden the application of spectral imaging techniques in various fields.

Purpose of the Study:

• To develop a cost-effective method for estimating near-infrared (NIR) reflectance from standard RGB images.
• To introduce a deep learning model, FUSE-Net, for enhanced spectral information recovery from RGB data.
• To evaluate the efficacy of the proposed G-RGB method and FUSE-Net for vegetation analysis.

Main Methods:

• Proposed a Green Normalized Difference Vegetation Index (GNDVI)-guided green channel adjustment method (G-RGB) to encode NIR-like information.
• Introduced FUSE-Net, a deep learning model integrating multi-scale convolutional layers and MLP-Mixer for spatial and spectral dependency modeling.
• Created a high-resolution RGB-HSI paired dataset of basil leaves for model evaluation.

Main Results:

• The G-RGB input significantly outperformed unmodified RGB across key metrics (MSE, PSNR, SCC, SSIM).
• FUSE-Net achieved the best performance when utilizing the G-RGB enhanced input.
• Ablation studies confirmed the model's ability to recover spectral information effectively.

Conclusions:

• The G-RGB method provides a viable approximation of NIR reflectance using only RGB images.
• This approach offers a cost-effective solution for vegetation analysis in scenarios where HSI systems are inaccessible.
• The developed method and model advance accessible spectral data analysis for precision imaging applications.