Multispectral imaging-based detection of apple bruises using segmentation network and classification model

Bruises can affect the appearance and nutritional value of apples and cause economic losses. Therefore, the accurate detection of bruise levels and bruise time of apples is crucial. In this paper, we proposed a method that combines a self-designed multispectral imaging system with deep learning to accurately detect the level and time of bruising on apples. To enhance the accuracy of extracting bruised regions with subtle features and irregular edges, an improved DeepLabV3+ was proposed. More specifically, depthwise separable convolution and efficient channel attention were employed, and the loss function was replaced with a focal loss. With these improvements, DeepLabV3+ achieved the maximum intersection over union of 95.5% and 91.0% for segmenting bruises on two types of apples in the test set, as well as maximum F1-score of 97.5% and 95.2%. In addition, the spectral data of the bruised regions were extracted. After spectral preprocessing, EfficientNetV2, DenseNet121, and ShuffleNetV2 were utilized to identify the bruise levels and times and DenseNet121 exhibited the best performance. To improve the identification accuracy, an improved DenseNet121 was proposed. The learning rate was adjusted using the cosine annealing algorithm, and squeeze-and-excitation attention mechanism and the Gaussian error linear unit activation function were utilized. Test set results demonstrated that the accuracies of the bruising levels were 99.5% and 99.1%, and those of the bruise time were 99.0% and 99.3%, respectively. This provides a new method for detecting bruise levels and bruised time on apples.