Large Language Models in Medical Image Analysis: A Systematic Survey and Future Directions

Area of Science:

• Artificial Intelligence
• Medical Imaging
• Natural Language Processing

Background:

• Large language models (LLMs) are increasingly integrated into medical image analysis.
• Their applications span report generation, disease classification, visual question answering, and segmentation.
• LLMs offer a novel approach to interpreting complex multimodal medical data.

Purpose of the Study:

• To survey and compile known applications of LLMs in medical image analysis.
• To highlight the potential benefits and critical challenges associated with LLM implementation.
• To outline future research directions and provide structured guidance for researchers and practitioners.

Main Methods:

• Introduction of the X-stage tuning framework for LLM fine-tuning (zero, one, and multi-stage).
• Comprehensive review of existing literature on LLMs in medical image analysis.
• Discussion of challenges including data sparsity, output hallucination, privacy, and knowledge updating.

Main Results:

• LLMs demonstrate diverse capabilities in medical image analysis tasks.
• The X-stage tuning framework provides a structured approach to fine-tuning LLMs based on task complexity and data availability.
• Identified key challenges require further investigation and innovative solutions.

Conclusions:

• LLMs present significant opportunities for advancing medical image analysis.
• Addressing challenges like data scarcity and privacy is crucial for widespread adoption.
• Future work should focus on integrating LLMs with decision support systems, multimodal learning, and federated learning.