A survey on advancements in image-text multimodal models: From general techniques to biomedical implementations

Area of Science:

• Artificial Intelligence
• Natural Language Processing
• Computer Vision

Background:

• Large Language Models (LLMs) have spurred advancements in Natural Language Processing (NLP).
• Image-text multimodal models integrate visual and textual data, gaining significant research attention.
• Existing surveys often overlook the influence of general models on domain-specific advancements.

Purpose of the Study:

• To review the technological evolution of image-text multimodal models.
• To analyze the impact of general model development on biomedical multimodal technologies.
• To identify and address challenges in general and domain-specific multimodal model development.

Main Methods:

• Reviewing technological evolution from feature spaces to large model architectures.
• Analyzing common components, tasks, and challenges of image-text multimodal models.
• Summarizing general model architectures, components, and data, with a focus on biomedical applications.

Main Results:

• General image-text multimodal technologies significantly promote biomedical multimodal progress.
• Biomedical datasets present unique importance and complexity for model development.
• Challenges are categorized into 2 external and 5 intrinsic factors, with proposed solutions.

Conclusions:

• Understanding the evolution of general models is crucial for domain-specific research.
• Targeted solutions are provided for challenges in multimodal model development and application.
• This survey guides future research directions in image-text multimodal AI.