M3SPCL: Multi-stage multi-grained multi-view supervised prototypical contrastive learning | JoVE Visualize

Area of Science:

Machine Learning
Computer Vision
Data Science

Background:

Existing multi-view learning methods often focus on instance-level modeling, neglecting crucial category-level semantics.
Current contrastive learning approaches for multi-view data can be computationally expensive due to exhaustive sample pairing.
A fundamental challenge lies in balancing category-aware effectiveness with computational efficiency in multi-view learning.

Purpose of the Study:

To introduce a novel framework, M3SPCL (Multi-stage Multi-grained Multi-view Supervised Prototypical Contrastive Learning), that addresses limitations in current multi-view learning techniques.
To enhance multi-view learning by effectively exploiting dual correlations at both instance and category levels.
To achieve superior performance and computational efficiency in multi-view data analysis.

Main Methods:

M3SPCL employs a multi-stage approach: early-stage paired view selection for efficient concatenation, intermediate-stage supervised and prototypical contrastive learning for multi-grained consistency, and late-stage adaptive decision fusion.
The framework integrates instance-level and category-level consistency enforcement through supervised and prototypical contrastive learning.
Prototype-based matching is utilized to reduce computational overhead by minimizing dense sample comparisons.

Main Results:

M3SPCL consistently outperforms state-of-the-art methods across eight diverse public multi-view benchmark datasets (image and text).
The proposed method achieves significant improvements in Accuracy (1.162%), Precision (1.703%), Recall (1.298%), and F-score (1.752%) over competitive baselines.
M3SPCL demonstrates substantial reductions in computational cost and memory usage compared to existing methods.

Conclusions:

M3SPCL offers a unified and effective framework for multi-view learning, successfully exploiting dual correlations at instance and category levels.
The proposed method achieves a strong balance between predictive effectiveness and computational efficiency.
M3SPCL represents a significant advancement in multi-view learning, particularly for datasets requiring both fine-grained and coarse-grained semantic understanding.