Self-Supervised Multimodal Learning: A Survey.

Area of Science:

• Artificial Intelligence
• Machine Learning
• Computer Vision

Background:

• Supervised multimodal learning has advanced significantly but relies heavily on costly human annotations.
• Large-scale unannotated data is abundant, making self-supervised learning a viable alternative.
• Self-supervised multimodal learning (SSML) leverages unlabeled data to bridge the gap.

Purpose of the Study:

• To provide a comprehensive review of the state-of-the-art in self-supervised multimodal learning.
• To identify and analyze key challenges in SSML.
• To survey existing solutions and applications of SSML.

Main Methods:

• Reviewing self-supervised objectives for learning multimodal representations from unlabeled data.
• Analyzing multimodal fusion strategies and model architectures.
• Examining pair-free learning for coarse- and fine-grained data alignment.

Main Results:

• Identified three core challenges: representation learning, modality fusion, and alignment.
• Detailed various self-supervised objectives and fusion techniques.
• Highlighted pair-free strategies for handling unaligned multimodal data.

Conclusions:

• SSML offers a promising direction for learning from raw multimodal data, reducing annotation dependency.
• Existing methods address key challenges, enabling diverse real-world applications.
• Future research should focus on further advancements in SSML techniques and applications.