A constrained singular value decomposition method that integrates sparsity and orthogonality.

Area of Science:

• Numerical Analysis
• Linear Algebra
• Machine Learning

Background:

• Singular Value Decomposition (SVD) is a fundamental matrix factorization technique.
• Standard SVD lacks mechanisms to enforce sparsity or orthogonality on singular vectors.
• Existing sparsification methods may not handle multiple constraints effectively.

Purpose of the Study:

• To propose a new sparsification method for SVD, termed Constrained Singular Value Decomposition (CSVD).
• To develop a method capable of incorporating multiple constraints, including sparsification and orthogonality, for singular vectors.
• To provide a robust and convergent algorithm for practical applications.

Main Methods:

• CSVD implements constraints as projections onto convex sets.
• Multiple constraints are integrated via projections onto the intersection of convex sets.
• An efficient algorithm for L1 and L2 norm projections is proposed and analyzed for convergence.

Main Results:

• CSVD is demonstrated to converge to a stable point with appropriate sparsification constants.
• The method is illustrated using simulated data, face image datasets, and a psychometric application.
• Performance is compared against standard SVD and a non-orthogonal sparsification method.

Conclusions:

• CSVD offers a flexible framework for incorporating diverse constraints into SVD.
• The proposed algorithm ensures convergence and is applicable to various data types and sizes.
• An R-package, csvd, is available for implementing the CSVD method.