Learning Feature-Sparse Principal Subspace.

Area of Science:

• Machine Learning
• Statistics
• Data Science

Background:

• Principal subspace estimation is crucial for dimension reduction when multiple principal components are of interest.
• Existing methods for feature-sparsity constrained PCA (FSPCA) often lack global convergence guarantees and rely on data distribution assumptions.

Purpose of the Study:

• Introduce two novel algorithms to address the feature-sparsity constrained PCA (FSPCA) problem for principal subspace estimation.
• Perform simultaneous feature selection and PCA, overcoming limitations of current optimization techniques.

Main Methods:

• Algorithm 1 globally solves FSPCA for low-rank covariance matrices.
• Algorithm 2 iteratively solves FSPCA for general covariance matrices using a designed proxy.
• Theoretical analysis provides data-dependent approximation bounds and convergence guarantees.

Main Results:

• Demonstrated global solvability of FSPCA for low-rank covariance.
• Established approximation bounds and convergence guarantees for both algorithms.
• Achieved exponential/posynomial approximation bounds for specific covariance spectrums.

Conclusions:

• The new algorithms provide efficient and effective solutions for FSPCA, enhancing principal subspace estimation.
• Experimental results confirm superior performance and efficiency compared to state-of-the-art methods on diverse datasets.