Multimodal Large Language Models in Medical Imaging: Current State and Future Directions

Area of Science:

• Artificial Intelligence in Medicine
• Medical Imaging Analysis
• Radiology Informatics

Background:

• Multimodal large language models (MLLMs) are emerging as powerful AI tools in medicine, especially radiology.
• They integrate large language models (LLMs) with diverse data, including clinical text and various radiological images (X-rays, CT, MRI).

Purpose of the Study:

• To review the current capabilities and limitations of MLLMs in medicine, with a focus on radiology.
• To outline key directions for future research and clinical integration of MLLMs.

Main Methods:

• Review of current MLLM applications in radiology, including report generation, visual question answering, and diagnostic support.
• Analysis of methods for multimodal integration and the impact of LLM advancements.

Main Results:

• MLLMs demonstrate potential for automating preliminary radiology reports and aiding diagnostics.
• Significant challenges include the scarcity of large-scale medical multimodal datasets, risk of hallucinated findings, lack of transparency, and high computational costs.

Conclusions:

• Future research should focus on region-grounded reasoning, developing robust foundation models, and establishing safe clinical integration strategies.
• Addressing current limitations is crucial for the widespread adoption of MLLMs in clinical radiology practice.