Sequential projection pursuit with kernel matrix update and symbolic model selection

Area of Science:

• Machine Learning
• Data Science
• Dimensionality Reduction

Background:

• Kernel methods are powerful for nonlinear data analysis.
• Feature extraction is crucial for improving classification performance.
• Existing projection pursuit methods often struggle with nonlinear structures.

Purpose of the Study:

• To develop a novel algorithm for generating reliable low-dimensional features with enhanced class separability.
• To adapt sequential projection pursuit for nonlinear feature extraction using kernel matrices.
• To enable automated, user-independent optimization of feature extraction parameters.

Main Methods:

• Utilizing an efficient sequential projection pursuit method.
• Implementing a new kernel matrix update scheme for nonlinear projections.
• Employing an adaptive kernel function to capture diverse data characteristics.
• Applying a holistic model selection procedure for parameter optimization.
• Solving the bi-level optimization problem using a hybrid evolutionary and gradient search approach.

Main Results:

• Demonstrated improved class separability in kernel-induced feature spaces.
• Successfully recovered multiple projections by gradually removing structure from residual dimensions.
• Showcased the effectiveness of the adaptive kernel function in unfolding data characteristics.
• Validated the algorithm's superiority over existing methods through benchmark evaluations.

Conclusions:

• The proposed method offers a robust approach for nonlinear dimensionality reduction and feature extraction.
• Automated optimization of projection index, dimensionality, and kernel parameters enhances data analysis.
• The algorithm provides a significant advancement in generating separable low-dimensional representations for classification tasks.