On Multi-Layer Basis Pursuit, Efficient Algorithms and Convolutional Neural Networks.

Area of Science:

• Signal Processing
• Machine Learning
• Computer Vision

Background:

• Parsimonious representations are fundamental in information modeling and processing.
• Existing Multi-Layer Convolutional Sparse Coding (ML-CSC) models inspire extensions to traditional sparse coding problems.

Purpose of the Study:

• To generalize the Basis Pursuit problem to a multi-layer setting with symbiotic sparse priors.
• To develop and analyze iterative algorithms for solving the multi-layer sparse coding problem.
• To demonstrate the connection between these algorithms and recurrent convolutional neural networks (CNNs).

Main Methods:

• Introduction of a generalized multi-layer Basis Pursuit problem.
• Development of Multi-Layer Iterative Soft Thresholding Algorithm (ML-ISTA) and its fast variant (ML-FISTA).
• Analysis of algorithm convergence properties and their implementation as recurrent CNNs.

Main Results:

• Demonstrated convergence of ML-ISTA and ML-FISTA algorithms.
• Established that these algorithms implement recurrent CNNs generalizing feed-forward architectures.
• Showcased improved performance of unfolded CNN architectures in supervised learning tasks.

Conclusions:

• The proposed multi-layer sparse coding framework offers a principled approach to constructing deep recurrent CNNs.
• Unfolded architectures derived from ML-ISTA/ML-FISTA improve performance over classical CNNs with constant parameters.
• This work bridges sparse representation theory and deep learning architectures.