Tensor product based 2-D correlation data preprocessing methods for Raman spectroscopy of Chinese handmade paper

Area of Science:

• Spectroscopy
• Materials Science
• Chemometrics

Background:

• Raman spectroscopy is crucial for analyzing materials like Chinese handmade paper.
• Preprocessing raw spectral data is essential for accurate analysis and machine learning model performance.
• Existing methods may not fully capture subtle spectral variations for complex sample analysis.

Purpose of the Study:

• Introduce novel preprocessing methods: cross-correlation (CCM) and two-dimensional correlation (TDCM) for Raman spectroscopy.
• Evaluate the effectiveness of CCM, TDCM (including TDSCM and TDACM), and baseline removal on machine learning model accuracy.
• Compare the performance of unsupervised (PCA) and supervised (SVM-LR, KNN, RF) models using different preprocessing techniques.

Main Methods:

• Developed and applied cross-correlation method (CCM) to expand spectral dimensions.
• Implemented two-dimensional correlation methods (TDCM), including synchronous (TDSCM) and asynchronous (TDACM), to create N×N spectral matrices.
• Utilized four machine learning models (PCA-LR, SVM-LR, KNN, RF) to analyze preprocessed Raman spectra of Chinese handmade paper.

Main Results:

• Principal Component Analysis (PCA) models showed near-perfect R-squared values across all data types.
• For supervised models (SVM-LR, KNN, RF), R-squared values improved sequentially: raw data < baseline removal < CCM < TDSCM < TDACM.
• The Two-Dimensional Asynchronous Correlation Method (TDACM) preprocessing led to R-squared values approaching 1 for KNN and RF models, a near 100% accuracy improvement.

Conclusions:

• CCM and TDCM are effective preprocessing techniques for Raman spectroscopy data analysis.
• TDACM preprocessing significantly enhances the accuracy of supervised machine learning models (KNN, RF) for Chinese handmade paper analysis.
• The enhanced accuracy of supervised models approaches that of unsupervised PCA, offering a powerful approach for spectral data interpretation.