Learning Deep Sparse Regularizers With Applications to Multi-View Clustering and Semi-Supervised Classification.

Area of Science:

• Machine Learning
• Optimization
• Deep Learning

Background:

• Sparsity-constrained optimization is crucial in machine learning applications like sparse coding and compressive sensing.
• Existing methods predominantly rely on hand-crafted sparse regularizers, limiting adaptability.
• There is a need for methods that learn sparse regularizers directly from data for specific tasks.

Purpose of the Study:

• To propose a deep sparse regularizer learning model that adaptively learns data-driven sparse regularizers.
• To establish the equivalence between sparse regularizer learning and parameterized activation function learning.
• To develop an end-to-end trainable neural network for learning sparse regularizers.

Main Methods:

• Developed a deep learning framework utilizing a neural network with multiple differentiable and reusable blocks.
• Incorporated learnable piecewise linear activation functions within each block, representing the sparse regularizer.
• Employed the proximal gradient algorithm and backpropagation for end-to-end model training.

Main Results:

• Demonstrated that sparse regularizer learning can be framed as learning parameterized activation functions.
• Successfully applied the framework to multi-view clustering and semi-supervised classification.
• Achieved superior performance compared to state-of-the-art multi-view learning models.

Conclusions:

• The proposed deep sparse regularizer learning framework effectively learns adaptive sparse regularizers from data.
• The model's architecture, based on learnable activation functions, enables end-to-end optimization.
• This approach offers a powerful new tool for tasks requiring latent compact representations and sparse solutions.