An Efficient Orthonormalization-Free Approach for Sparse Dictionary Learning and Dual Principal Component Pursuit

Area of Science:

• Optimization
• Machine Learning
• Data Analysis

Background:

• Sparse dictionary learning (SDL) is a fundamental technique in data analysis.
• Recently, Lm-norm maximization (m ≥ 3) has emerged as a method for solving SDL, transforming it into an optimization problem with orthogonality constraints.

Purpose of the Study:

• To propose an Lm-norm maximization model for dual principal component pursuit (DPCP).
• To develop an efficient algorithm for solving the proposed optimization models.

Main Methods:

• Formulated an Lm-norm maximization model for DPCP, leveraging similarities with SDL.
• Introduced a smooth unconstrained exact penalty model equivalent to the Lm-norm maximization model.
• Developed an efficient first-order algorithm (PenNMF) for the penalty model, proving its global convergence.

Main Results:

• The proposed penalty model is equivalent to the Lm-norm maximization model.
• The PenNMF algorithm demonstrates high efficiency in solving Lm-norm maximization with orthogonality constraints.
• Experimental results show PenNMF outperforms state-of-the-art algorithms.

Conclusions:

• The PenNMF algorithm offers an efficient solution for Lm-norm maximization problems in DPCP and SDL.
• The study advances optimization techniques for representation learning with orthogonality constraints.