Deep Clustering: On the Link Between Discriminative Models and K-Means

Area of Science:

• Machine Learning
• Deep Learning
• Unsupervised Learning

Background:

• Discriminative models, commonly used in deep clustering, often outperform generative approaches.
• These models typically employ logistic regression posteriors and parameter regularization, similar to supervised learning.

Purpose of the Study:

• To demonstrate the equivalence between recent discriminative deep clustering models and K-means.
• To introduce a novel deep K-means algorithm based on theoretical findings.

Main Methods:

• Theoretical analysis connecting discriminative objective functions (mutual information maximization) to K-means loss.
• Utilizing an approximate alternating direction method (ADM) for optimization.

Main Results:

• Proved that L2 regularized mutual information maximization is equivalent to minimizing a soft, regularized K-means loss for logistic regression posteriors.
• Developed a new soft and regularized deep K-means algorithm.

Conclusions:

• State-of-the-art discriminative deep clustering models can be viewed as variants of K-means.
• The proposed deep K-means algorithm achieves competitive results on image clustering benchmarks.