A hybrid variable selection strategy based on continuous shrinkage of variable space in multivariate calibration.

Area of Science:

• Chemometrics
• Spectroscopy
• Data Science

Background:

• High-dimensional near-infrared (NIR) spectral data analysis requires effective variable selection to enhance model predictive capabilities.
• Existing variable selection methods often face limitations like overfitting, time inefficiency, and high computational demands with numerous variables.

Purpose of the Study:

• To propose and evaluate a novel hybrid variable selection strategy for high-dimensional NIR spectral datasets.
• To leverage the strengths of Variable Combination Population Analysis (VCPA), Genetic Algorithm (GA), and Iteratively Retaining Informative Variables (IRIV) while mitigating their individual drawbacks.

Main Methods:

• A hybrid strategy was developed, starting with a modified VCPA for continuous variable space shrinkage.
• Further optimization was achieved using Iteratively Retaining Informative Variables (IRIV) and a Genetic Algorithm (GA).
• The strategy was tested on three NIR datasets against established methods like CARS, GA-iPLS, and VIP-GA.

Main Results:

• The proposed VCPA-based hybrid strategies (VCPA-GA and VCPA-IRIV) demonstrated significant improvements in model prediction performance.
• The modified VCPA step proved effective in filtering uninformative variables.
• Comparative analysis showed superior results compared to other tested variable selection methods.

Conclusions:

• The VCPA-based hybrid strategy offers a robust and promising approach for variable selection in NIR spectral analysis.
• This method effectively addresses the challenges posed by high-dimensional datasets.
• The developed strategies enhance predictive accuracy and model efficiency.