Dual-channel self-supervised multi-task learning for spectral detection of soluble solids content and firmness in...

To reduce the cost of labeled data in fruit quality detection, this study proposes a deep learning framework combining multi-source spectral data, self-supervised learning (SSL), and multi-task learning (MTL). In this study, spectral data in the visible/near-infrared (Vis/NIR) and near-infrared (NIR) ranges were collected, and soluble solids content (SSC) and firmness (FI) were measured of Korla fragrant pear. The single-task and multi-task learning convolutional neural networks (STL-CNN and MTL-CNN) were built to predict the SSC and FI of Korla fragrant pear using Vis/NIR, NIR, and fused spectra respectively. Meanwhile, an SSL strategy was introduced to utilize unlabeled spectral data for knowledge transfer and to fine-tuning the CNN model parameters using limited labeled data. The results demonstrated that the prediction performance of MTL-CNN was superior to STL-CNN. The correlation coefficient of prediction (R_p) and root mean square error (RMSEP) of MTL-CNN (Vis/NIR-NIR) in SSC prediction were 0.9576 and 0.3657 %, respectively, while those in FI prediction were 0.9297 and 0.9941 N, respectively. Both models outperformed the MTL-CNN built using single spectral data (Vis/NIR and NIR). In addition, SSL significantly improved the CNN prediction performance when trained on small sample sizes. The SSL-MTL-CNN (Vis/NIR-NIR) was trained using only 700 samples, yet it achieved R_p values exceeding 0.9 for both FI and SSC. In conclusion, the integration of multi-source spectral data with an SSL-MTL strategy could enhance the prediction performance of the quality in Korla fragrant pear and provide an effective method for the simultaneous detection of multiple fruit quality attributes.