M3SPCL: Multi-stage multi-grained multi-view supervised prototypical contrastive learning

Area of Science:

• Machine Learning
• Computer Vision
• Data Science

Background:

• Existing multi-view learning methods often focus on instance-level modeling, neglecting crucial category-level semantics.
• Current contrastive learning approaches for multi-view data can be computationally expensive due to exhaustive sample pairing.
• A fundamental challenge lies in balancing category-aware effectiveness with computational efficiency in multi-view learning.

Purpose of the Study:

• To introduce a novel framework, M3SPCL (Multi-stage Multi-grained Multi-view Supervised Prototypical Contrastive Learning), that addresses limitations in current multi-view learning techniques.
• To enhance multi-view learning by effectively exploiting dual correlations at both instance and category levels.
• To achieve superior performance and computational efficiency in multi-view data analysis.

Main Methods:

• M3SPCL employs a multi-stage approach: early-stage paired view selection for efficient concatenation, intermediate-stage supervised and prototypical contrastive learning for multi-grained consistency, and late-stage adaptive decision fusion.
• The framework integrates instance-level and category-level consistency enforcement through supervised and prototypical contrastive learning.
• Prototype-based matching is utilized to reduce computational overhead by minimizing dense sample comparisons.

Main Results:

• M3SPCL consistently outperforms state-of-the-art methods across eight diverse public multi-view benchmark datasets (image and text).
• The proposed method achieves significant improvements in Accuracy (1.162%), Precision (1.703%), Recall (1.298%), and F-score (1.752%) over competitive baselines.
• M3SPCL demonstrates substantial reductions in computational cost and memory usage compared to existing methods.

Conclusions:

• M3SPCL offers a unified and effective framework for multi-view learning, successfully exploiting dual correlations at instance and category levels.
• The proposed method achieves a strong balance between predictive effectiveness and computational efficiency.
• M3SPCL represents a significant advancement in multi-view learning, particularly for datasets requiring both fine-grained and coarse-grained semantic understanding.