Standardization of near infrared spectroscopies via sample spectral correlation equalization

A novel method for near-infrared (NIR) spectroscopy spectra standardization is presented. NIR spectroscopies have been widely used in analytical chemistry, and many methods have been developed for NIR spectra standardization. To establish a robust standardization transformation, most existing methods require spectral data sets from both primal and secondary instruments for 1-1 correspondence validation. However, this limits the usage of standardization methods. This paper investigates an interesting issue, "Can spectra data in sets be arbitrarily order?" and further develops a completely different approach from existing methods in view of statistical signal processing. The key idea is to first compensate for the distortion along the wavelength and intensity of the spectra, and then transfer the second order statistic (2OS) from the primal spectra to the secondary spectra via data sphering and an inverse sphering transform so that the 2OS can be estimated regardless of the sample statistic order. To further demonstrate how the developed method can extend the usage of the NIR spectra standardization, several application-driven experiments on classification and regression are conducted for demonstration, and a comparison to the piecewise direct standardization (PDS) is also studied.