Elastic-Net Prefiltering for Two-Class Classification.

Area of Science:

• Machine Learning
• Data Science
• Computational Statistics

Background:

• Noisy two-class classification problems present significant challenges in pattern recognition and data analysis.
• Existing methods often struggle with generalization and computational efficiency when dealing with noisy data.

Purpose of the Study:

• To propose a robust two-stage algorithm for constructing linear-in-the-parameter models for noisy two-class classification.
• To enhance model generalization capability and achieve sparse classifier construction.

Main Methods:

• A two-stage approach involving a prefiltering signal generation and a sparse classifier construction.
• The prefiltering stage utilizes an elastic-net model identification algorithm with singular value decomposition (SVD) and particle swarm optimization (PSO) for regularization parameter tuning.
• The classifier construction employs orthogonal forward regression (OFR) with the D-optimality algorithm.

Main Results:

• The proposed method analytically computes the leave-one-out (LOO) misclassification rate with minimal computational cost due to orthogonality.
• The prefiltered signal serves as the desired output for the second stage, leading to a sparse linear classifier.
• Extensive simulations demonstrate the competitiveness of the approach for noisy data classification.

Conclusions:

• The developed two-stage algorithm offers a competitive and efficient solution for noisy two-class classification problems.
• The combination of elastic-net, SVD, PSO, and OFR provides a powerful framework for robust model construction.
• The method effectively balances model generalization and sparsity in the presence of data noise.