Learning Cross-Modal Common Representations by Private-Shared Subspaces Separation.

Area of Science:

• Computer Science
• Artificial Intelligence
• Machine Learning

Background:

• Correlating heterogeneous data (e.g., images and text) is challenging due to inconsistent distributions.
• Existing common representation learning methods often use a two-stage approach and neglect interference within modalities.

Purpose of the Study:

• To propose a novel model that explicitly models interference within each modality to improve common representation learning.
• To enhance cross-modal retrieval by learning more effective common representations.

Main Methods:

• Introduced the private-shared subspaces separation (P3S) model for one-stage joint learning.
• P3S partitions representations into common (shared subspace) and private (private subspaces) components.
• Employs orthogonality constraints between shared and private subspaces to exclude modality-specific interference.

Main Results:

• The P3S method demonstrates significant advantages in cross-modal retrieval.
• Achieved superior performance compared to 15 state-of-the-art methods on four widely used datasets.
• Explicitly modeling interference leads to more effective common representations.

Conclusions:

• The proposed P3S model effectively addresses the challenge of interference in cross-modal representation learning.
• This one-stage joint learning approach with subspace separation offers a more effective solution for cross-modal retrieval tasks.