Improving prediction model robustness with virtual sample construction for near-infrared spectra analysis.

Area of Science:

• Analytical Chemistry
• Spectroscopy
• Chemometrics

Background:

• Near-infrared spectroscopy (NIRS) models suffer from poor robustness and generalization when dealing with limited or imbalanced sample data.
• Insufficient or imbalanced sample sizes hinder the development of reliable NIRS analytical models, particularly when sample acquisition is challenging.
• Virtual sample construction is a key strategy to address data imbalance and enhance model performance in NIRS.

Purpose of the Study:

• To evaluate three virtual spectrum construction strategies: Synthetic Minority Oversampling Technique (SMOTE), Adaptive Synthetic Sampling (ADASYN), and Deep Convolutional Generative Adversarial Network (DCGAN).
• To assess the effectiveness of these strategies in improving the robustness and generalization ability of NIRS models with imbalanced datasets.
• To compare the performance of SMOTE, ADASYN, and DCGAN in enhancing discriminant model accuracy using NIRS spectral data.

Main Methods:

• Exploration of SMOTE, ADASYN, and DCGAN for virtual spectrum generation to balance spectral datasets.
• Application of these strategies to melamine and Yali pear spectral datasets.
• Construction of discriminant models using Partial Least Squares-Discriminant Analysis (PLS-DA) and evaluation of accuracy via Correct Recognition Rate (CRR).

Main Results:

• All tested strategies (SMOTE, ADASYN, DCGAN) improved the global Correct Recognition Rate (CRR_glob).
• SMOTE and ADASYN improved CRR_glob but decreased the CRR for minority class samples (CRR_min).
• DCGAN demonstrated superior performance by improving CRR_glob, CRR_maj, and CRR_min, with results indicating high robustness and reliability.

Conclusions:

• Deep Convolutional Generative Adversarial Network (DCGAN) is a reliable and practical method for generating virtual spectral samples.
• DCGAN effectively enhances the robustness and generalization ability of NIRS models, particularly in scenarios with insufficient or imbalanced sample data.
• The study highlights DCGAN as a promising approach for overcoming data limitations in NIRS analysis, outperforming traditional methods like SMOTE and ADASYN.