Toward Generalized Multistage Clustering: Multiview Self-Distillation.

Area of Science:

• Machine Learning
• Data Science
• Computer Vision

Background:

• Multiview clustering (MVC) methods often struggle with noisy data and inadequate feature learning.
• Existing MVC paradigms can generate overconfident pseudo-labels, leading to inaccurate predictions and accumulated bias.
• There is a need for methods that can correct pseudo-label misdirection in multistage clustering.

Purpose of the Study:

• To propose a novel multistage deep multiview clustering framework, DistilMVC.
• To introduce multiview self-distillation to correct overconfident pseudo-labels and improve generalization.
• To enhance the robustness and predictive capabilities of clustering models.

Main Methods:

• Explores common semantics across multiple views using contrastive learning at different feature hierarchies.
• Obtains pseudo-labels by maximizing mutual information between views.
• Employs a teacher network to distill pseudo-labels into dark knowledge, guiding a student network.

Main Results:

• The proposed DistilMVC framework demonstrates improved clustering performance.
• The method effectively alleviates the impact of overconfident pseudo-labels.
• Experiments show superior results compared to state-of-the-art methods on real-world datasets.

Conclusions:

• DistilMVC offers a robust solution for multistage deep multiview clustering.
• Self-distillation of dark knowledge enhances model accuracy and generalization.
• The framework effectively addresses limitations of existing MVC techniques.