Optimization of diffuse reflectance spectroscopy measurements for direct and rapid screening of pesticides: A case...

Although diffuse reflectance spectroscopy (DRS) measurements can be collected rapidly and simultaneously, the resulting datasets are imbalanced and redundant due to the highly correlated spectral features collected on relatively few samples. Consequently, modelling these datasets using machine learning (ML) techniques is challenging and necessitates longer training times and more computational resources. Furthermore, models developed with such data are frequently prone to overfitting, resulting in promising but often non-reproducible results. We demonstrate the advantage of using an eigenvector decomposition principal component analysis (PCA) in reducing the dimensionality and data mining of DRS measurements in the short near-infrared region (750-900 nm). A total of 547 DRS measurements consisting of 151 wavelengths were acquired from spinach samples sprayed with two different pesticides and control samples. The measurements were later preprocessed with a Savitzky-Golay filter and multiplicative scatter analysis. After performing PCA on the preprocessed data, two principal components (PCs) that explained 77% of the cumulative variance and maximized the interclass variation were extracted and used as inputs to three ML models namely; artificial neural networks, support vector machine and random forest, to classify the samples. Re-sampling was used to tune the models and avoid overfitting. The performance of the models was compared using raw DRS data, pre-processed (PP) DRS data, and PCs data. The results show that pesticide classification using PCs data requires the least amount of training time (average 2.4 s) for all the models, and achieves 100% classification accuracy. In addition, it was observed that spectral data pre-processing improves accuracy and training time when compared to using raw spectral data. These findings are particularly encouraging since they demonstrate the possibility of developing rapid and accurate classification models for screening pesticide residues in fresh produce based on DRS measurements with minimal computational resources.