Convolutional Dictionary Learning: Acceleration and Convergence | JoVE Visualize

Area of Science:

Computer Vision
Image Processing
Machine Learning

Background:

Convolutional Dictionary Learning (CDL) is vital for image processing and computer vision.
Existing efficient CDL algorithms often use Augmented Lagrangian (AL) or Alternating Direction Method of Multipliers (ADMM), which require complex parameter tuning and can struggle with convergence for non-convex problems.
Parameter tuning for AL/ADMM methods is data-dependent and critical for convergence, posing practical challenges.

Purpose of the Study:

To introduce a novel, practically feasible, and convergent Block Proximal Gradient method using a Majorizer (BPG-M) for Convolutional Dictionary Learning (CDL).
To investigate the performance of BPG-M-based CDL with various block updating schemes and majorization matrix designs.
To enhance BPG-M's efficiency through momentum coefficients and restarting techniques, and to evaluate its effectiveness in image processing tasks.

Main Methods:

Developed a Block Proximal Gradient method using a Majorizer (BPG-M) for CDL.
Explored different block updating strategies and majorization matrix designs within the BPG-M framework.
Incorporated momentum coefficients and restarting techniques to accelerate convergence.
Integrated a boundary artifacts removal operator into the learning model for all investigated methods.

Main Results:

BPG-M demonstrates stable convergence to lower objective values compared to state-of-the-art ADMM algorithms, without requiring any parameter tuning.
The multi-block updating scheme of BPG-M offers lower memory requirements and avoids polynomial computational complexity, making it suitable for large datasets on single-threaded systems.
Image denoising experiments show superior performance of filters learned by BPG-M-based CDL, especially under strong additive white Gaussian noise, compared to ADMM-trained filters.

Conclusions:

The proposed BPG-M method provides a robust and efficient alternative to ADMM for CDL, particularly advantageous for large-scale problems and when parameter tuning is undesirable.
BPG-M offers improved stability and solution quality in CDL applications.
The method shows significant promise for practical applications in image processing, such as noise reduction.