A novel variable selection method based on combined moving window and intelligent optimization algorithm for variable...

Area of Science:

• Chemometrics
• Spectroscopy
• Machine Learning

Background:

• Variable selection is crucial for building robust chemometric models.
• Existing methods may suffer from local extrema and overfitting.
• Optimizing spectral interval selection requires efficient algorithms.

Purpose of the Study:

• To introduce a new wavelength selection algorithm, Variable Dimension Particle Swarm Optimization-Combined Moving Window (VDPSO-CMW).
• To enhance spectral data analysis by improving variable selection accuracy and model generalization.
• To provide an efficient and robust method for optimizing spectral intervals.

Main Methods:

• The proposed VDPSO-CMW algorithm combines the strengths of Combined Moving Window (CMW) and Variable Dimension Particle Swarm Optimization (VDPSO).
• CMW allows for automatic optimization of spectral interval width and number with overlapping windows.
• VDPSO enhances Particle Swarm Optimization (PSO) by enabling multi-dimensional search and mitigating local extrema and overfitting risks.

Main Results:

• VDPSO-CMW was compared against four established variable selection algorithms (BOSS, VCPA, iVISSA, IRF) on three Near-Infrared (NIR) datasets (corn, beer, fuel).
• The VDPSO-CMW algorithm demonstrated superior performance in variable selection compared to the other methods.
• The algorithm effectively optimized spectral intervals, leading to improved model performance.

Conclusions:

• The VDPSO-CMW algorithm offers a significant advancement in wavelength selection for spectroscopic data analysis.
• This method provides a more robust and accurate approach to variable selection, reducing overfitting.
• Freely available Matlab codes facilitate the implementation and adoption of this advanced technique.