MedSegAgent: A Universal and Scalable Multi-Agent System for Instructive Medical Image Segmentation.

Area of Science:

• Medical Imaging
• Artificial Intelligence
• Computational Biology

Background:

• Current medical image segmentation methods lack a universal framework, scalability, and user-friendly interfaces.
• Diverse modalities and anatomical targets pose challenges for existing segmentation solutions.
• Non-expert users struggle with complex model selection and deployment in clinical settings.

Purpose of the Study:

• To introduce MedSegAgent, a universal and scalable multi-agent system for instructive medical image segmentation.
• To address limitations in existing medical image segmentation frameworks, including adaptability and usability.
• To bridge the gap between natural language queries and complex model selection for segmentation tasks.

Main Methods:

• Developed MedSegAgent, a multi-agent system with five agents: query parsing, modality filtering, anatomical filtering, label selection, and execution.
• Utilized 23 diverse datasets and pre-trained models for comprehensive segmentation capabilities.
• Implemented coarse-to-fine filtering agents to identify relevant datasets and label values based on user queries.

Main Results:

• MedSegAgent accurately identified matching datasets and labels in 94.27% of queries.
• A suitable match was located in 99.03% of all queries, demonstrating high system reliability.
• The system simplified model selection while maintaining high performance across various segmentation tasks.

Conclusions:

• MedSegAgent provides a universal and scalable solution for diverse medical image segmentation tasks.
• The system enhances usability for non-expert users by processing natural language requests.
• MedSegAgent effectively integrates user-friendly queries with complex model selection and deployment processes.