Early apple moldy core classification via multi-modal sensing and SE-ResNet18

Area of Science:

• Agricultural Science
• Computer Vision
• Spectroscopy

Background:

• Apple moldy core disease significantly impacts postharvest quality.
• Early internal lesions are undetectable by visual inspection, necessitating advanced diagnostic methods.

Purpose of the Study:

• To develop an efficient, non-destructive method for early diagnosis of apple moldy core disease.
• To fuse Visible-Near Infrared Spectroscopy (Vis-NIR) and Electronic Nose (E-nose) data for improved disease classification.

Main Methods:

• 1D time-series and spectral data were transformed into image representations using Gramian Angular Field (GAF), Markov Transition Field (MTF), and Recurrence Plot (RP) techniques.
• A two-branch SE-ResNet18 deep learning model with a channel attention mechanism was employed for multimodal fusion and classification.
• The SE attention module was utilized for feature calibration and modal balance.

Main Results:

• The multimodal fusion model achieved a classification accuracy of 95.93%, outperforming single-modal approaches.
• Ablation studies confirmed the critical role of the SE attention module in enhancing feature representation and modal balance.
• Information entropy analysis demonstrated improved discriminability of features extracted by the proposed method.

Conclusions:

• Multimodal data fusion effectively complements information from different sources for accurate disease detection.
• The proposed method offers a reliable and promising solution for the non-destructive detection of early diseases in agricultural products.
• This approach has significant potential for practical application and widespread adoption in the agricultural industry.