A novel interval sparse evolutionary algorithm for efficient spectral variable selection.

Area of Science:

• Chemometrics
• Machine Learning
• Optimization

Background:

• Effective spectral analysis relies on selecting optimal spectral variables to reduce dimensionality and enhance model performance.
• Variable selection in spectral analysis is an NP-hard problem, with existing algorithms struggling to balance effectiveness and computational speed.
• A need exists for advanced methods to address the limitations of current spectral variable selection techniques.

Purpose of the Study:

• To propose a novel Interval Sparse Evolutionary Algorithm (ISEA) for large-scale sparse multi-objective optimization in spectral variable selection.
• To improve prediction accuracy by modeling spectral data with fewer, more informative variables.
• To develop an algorithm that enhances both the effectiveness and speed of spectral variable selection.

Main Methods:

• Modeled variable selection as a large-scale sparse multi-objective optimization problem.
• Developed the Interval Sparse Evolutionary Algorithm (ISEA), integrating interval partial least squares (iPLS) with evolutionary algorithms.
• Incorporated a sparse population initialization strategy (SPIS) and a regional sparse evolution strategy (RSES) with a roulette probability mechanism to prioritize informative variables and regions.

Main Results:

• The proposed ISEA demonstrated superior performance compared to nine state-of-the-art methods on corn oil, soil, and diesel fuel datasets.
• ISEA achieved a balance between the effectiveness of variable selection and computational running speed.
• The algorithm successfully reduced prediction errors by selecting fewer, more relevant spectral variables.

Conclusions:

• The Interval Sparse Evolutionary Algorithm (ISEA) offers a significant advancement in spectral variable selection.
• ISEA's effectiveness and speed make it a valuable tool for chemometrics and other large-scale sparse problems.
• The algorithm shows broad applicability beyond spectral analysis, including critical node detection and neural network training.