Multi-View Clustering With Cauchy-Schwarz Mutual Information Maximin

Area of Science:

• Machine Learning
• Information Theory
• Computer Vision

Background:

• Deep multi-view clustering (MVC) utilizes information bottleneck (IB) for optimizing data compression and feature preservation.
• Existing IB-based MVC methods often rely on variational inference for mutual information (MI) estimation, leading to potential errors and instability.
• This limits the effectiveness of deep MVC in capturing complex multi-view relationships.

Purpose of the Study:

• To develop a novel method for direct and stable mutual information estimation in deep multi-view clustering.
• To enhance the performance of multi-view clustering by improving the information bottleneck principle.
• To introduce a new approach for modeling multi-view information and cross-view complementarity.

Main Methods:

• Proposed a Cauchy-Schwarz Mutual Information Maximin (CS-MIM) method for direct MI estimation using closed-form expressions.
• Introduced a non-parametric MI estimation using Cauchy-Schwarz (CS) divergence with multi-kernel Gram matrices, avoiding variational inference errors.
• Developed a MI maximin mechanism with analytical gradients for effective data compression and feature preservation within the IB framework.
• Designed a cross-view adaptive attention (CAA) mechanism guided by CS divergence-based MI to capture inter-view complementarity.

Main Results:

• The CS-MIM method directly estimates MI without relying on variational inference, ensuring stability and accuracy.
• The proposed approach effectively compresses multi-view data while preserving essential features through an analytical gradient-based MI maximin mechanism.
• The cross-view adaptive attention mechanism successfully captures complementary information across different views.
• Empirical evaluations on 12 datasets show that CS-MIM significantly outperforms state-of-the-art methods in multi-view clustering.

Conclusions:

• The CS-MIM method offers a more stable and accurate approach to mutual information estimation for deep multi-view clustering.
• This novel method enhances the information bottleneck principle by providing direct MI estimation and analytical gradients.
• The study demonstrates the effectiveness of CS-MIM in improving multi-view clustering performance, particularly in capturing cross-view relationships.