Non-Destructive Detection and Grading of Plum Quality Based on Multimodal Data

Assessing plum quality solely based on the external appearance of the peel may lead to inaccurate results. This paper proposes a multimodal data fusion technique based on deep learning, which evaluates plum quality by fusing color image data and spectral data of plums, thereby enabling plum grading. The method utilizes the Visual Geometry Group 16-layer network (VGG16) to extract plum image features, and a 1D Convolutional Neural Network (1D-CNN) to extract near-infrared spectral data of plums, subsequently classifying plum quality through the network's fully connected layers and output layer. Each modality independently extracts features: color images provide external color information, while visible and near-infrared spectroscopy (wavelength range 350-1700 nm) captures surface and internal chemical composition spectral properties. The multimodal preprocessing and feature extraction process creates an optimal comprehensive information representation for plum quality assessment within the feature space. By processing these integrated features through a fully connected neural network, the classification accuracy of plum quality reaches 100%, significantly outperforming single-modal methods (color imaging: 85.71%; spectroscopy: 83.33%). The performance comparison analysis between multimodal fusion and single-modal methods further confirms the robustness and applicability of the multimodal fusion approach, providing new technical means for non-destructive detection and grading of plum quality. This method can simultaneously detect internal and external quality indicators, compensating for the limitations of traditional single-dimensional detection.