Grape sugar content prediction with multispectral alignment and improved residual network

Sugar content is a crucial indicator of grape ripeness and grading, and developing non-contact and non-destructive sugar content detection devices is essential for grape-picking robots and sorting platforms. Spectroscopy, which can detect the chemical composition of grapes, has become a key technology for developing non-destructive testing devices. In this paper, we collected 2,880 randomly labeled multispectral images of Sunshine Rose grapes with a Changguang Yuchen MS600 PRO multispectral camera and measured the sugar content (in Brix values) of the labeled grapes with a handheld refractometer, using data exclusively from this grape variety. To address noise and misalignment issues in the multispectral images, we proposed preprocessing methods including Gaussian denoising and ECC (Enhanced Correlation Coefficient) algorithm registration. Based on a ResNet-50 residual network, we constructed a grape sugar content prediction regression model Improved-Res with SE (Squeeze-and-Excitation) attention modules, DSC (Depthwise Separable Convolutions), and Inception modules. The model's performance was evaluated by MSE (Mean Squared Error), MAE (Mean Absolute Error), and R² (R-Square) metrics. We compared the performance of four feature extraction methods combined with four traditional machine learning models, as well as seven deep learning models. The results showed that among traditional machine learning methods, the combination of color histogram feature extraction and the XGBoost regression achieved the best performance, with MSE, MAE, and R² of 1.35, 0.90 Brix, and 0.78, respectively. Among deep learning methods, the ResNet-50 model demonstrated the best performance, with MSE, MAE, and R² of 0.95, 0.96 Brix, and 0.84, respectively. Effective improvements of SE attention module, depthwise separable convolutions, and Inception module in the ResNet-50 model was confirmed through ablation experiments: the proposed Improved-Res model achieved MSE, MAE, and R² of 0.49, 0.55 Brix, and 0.92, respectively, which significantly outperformed traditional machine learning methods and classical deep learning models.