Deep unsupervised feature selection by discarding nuisance and correlated features.

Area of Science:

• Machine Learning
• Data Science
• Computer Science

Background:

• Modern datasets frequently contain correlated and nuisance features, complicating data analysis.
• Nuisance features, unrelated to underlying data structures, can hinder model performance.
• Existing methods often struggle to address both correlated and nuisance features simultaneously.

Purpose of the Study:

• To develop a novel unsupervised feature selection method capable of handling both correlated and nuisance features.
• To improve the accuracy and efficiency of data analysis and clustering in complex datasets.
• To provide a robust approach for identifying and selecting relevant features.

Main Methods:

• A fully differentiable unsupervised feature selection approach is proposed, integrating the Laplacian score criterion.
• An autoencoder architecture is employed to manage correlated features by reconstructing data from selected features.
• A concrete layer is utilized to control the number of selected features, simplifying optimization.

Main Results:

• The proposed method effectively identifies and avoids nuisance features by computing the Laplacian on selected features.
• Experimental results on real-world datasets show superior performance compared to methods addressing only correlated or nuisance features.
• State-of-the-art clustering results were achieved, demonstrating the method's efficacy.

Conclusions:

• The developed unsupervised feature selection technique offers a significant advancement for analyzing datasets with complex feature relationships.
• The integration of Laplacian score and autoencoder architectures provides a powerful solution for feature selection challenges.
• The method's ability to handle both correlated and nuisance features leads to improved data representation and downstream task performance.