Foundation Models Defining a New Era in Vision: A Survey and Outlook.

Area of Science:

• Computer Vision
• Artificial Intelligence
• Multimodal Learning

Background:

• Understanding complex visual scenes requires integrating information from various modalities like language, audio, and depth.
• Foundation models bridge modalities and large datasets, enabling contextual reasoning and generalization.
• These models allow prompt-based modification without retraining, enhancing flexibility.

Purpose of the Study:

• To provide a comprehensive review of emerging foundation models in computer vision.
• To detail their architectural designs, training objectives, and prompting patterns.
• To discuss current challenges and future research directions.

Main Methods:

• Systematic review of literature on multimodal foundation models.
• Analysis of architectural designs, training methodologies (contrastive, generative), and pre-training datasets.
• Categorization of prompting patterns (textual, visual, heterogeneous).

Main Results:

• Detailed overview of foundation model architectures for integrating vision, text, audio, and other modalities.
• Summary of various training objectives and pre-training strategies.
• Comprehensive analysis of prompting techniques and their applications.

Conclusions:

• Foundation models represent a significant advancement in computer vision, enabling sophisticated scene understanding.
• Key challenges include evaluation difficulties, real-world understanding gaps, bias, and interpretability.
• Future research should address these challenges to unlock the full potential of foundation models.