Multimodal Artificial Intelligence in Tissue Diagnostics: Vision-Language Models and the Future of Computational...

Area of Science:

• Multimodal artificial intelligence (AI)
• Computational pathology
• Vision-language models (VLMs)

Background:

• VLMs bridge visual histology from whole slide images (WSIs) with text data like pathology reports.
• This technology enables new approaches in computational pathology by integrating diverse data sources.

Purpose of the Study:

• To review the technical foundations, applications, evaluation, and deployment of pathology VLMs.
• To summarize current use cases and future considerations for VLMs in pathology.

Main Methods:

• Review of technical foundations including image encoders, text encoders/LLMs, and multimodal alignment.
• Summary of applications: image-text retrieval, classification, VQA, localization, anomaly detection, report generation, and agentic workflows.
• Analysis of evaluation strategies and deployment considerations for pathology VLMs.

Main Results:

• Pathology VLMs support diverse applications like image-text retrieval and report generation.
• Key enabling technologies include encoders, LLMs, and multimodal alignment strategies.
• Significant barriers remain in evaluation, robustness, and clinical translation.

Conclusions:

• VLMs show promise for augmenting, not replacing, pathologists.
• Responsible development requires collaboration and addressing challenges in validation, integration, and accountability.
• Ensuring safe, interpretable, and clinically meaningful improvements in pathology practice is crucial.