Discriminative and Uncorrelated Feature Selection With Constrained Spectral Analysis in Unsupervised Learning

Area of Science:

• Machine Learning
• Data Science
• Computer Vision

Background:

• Unsupervised feature extraction often uses uncorrelated constraints, but this can lead to trivial solutions with high-dimensional data.
• Existing methods struggle with the singularity of scatter matrices in high-dimensional spaces.

Purpose of the Study:

• To propose a novel regularized regression model for unsupervised feature selection.
• To address the limitations of existing methods in handling high-dimensional data and trivial solutions.
• To develop a method that extracts low-redundant and discriminative features.

Main Methods:

• A regularized regression model with a generalized uncorrelated constraint is proposed.
• Constrained spectral analysis is employed to capture local cluster structure using Must-Links and Cannot-Links.
• The proposed Discriminative and Uncorrelated Feature Selection with Constrained Spectral Analysis (DUCFS) utilizes σ-norm regularization.

Main Results:

• The DUCFS model successfully explores low-redundant and discriminative features.
• The method avoids trivial solutions and simplifies the optimization process.
• Experiments demonstrate the effectiveness of DUCFS compared to state-of-the-art approaches on benchmark datasets.

Conclusions:

• The proposed DUCFS method offers an effective approach for unsupervised feature selection, particularly for high-dimensional data.
• The novel constrained spectral analysis and regularization techniques contribute to improved feature extraction.
• The model exhibits fast convergence due to its flexible gradient and global differentiability.