MAP: evaluation and multi-agent enhancement of large language models for inpatient pathways

Area of Science:

• Artificial Intelligence in Medicine
• Clinical Decision Support Systems
• Healthcare Informatics

Background:

• Inpatient clinical decision-making relies on comprehensive patient data, but medical large language model (LLM) research is hindered by limited datasets.
• Current medical LLM benchmarks often focus on question-answering and exams, neglecting the complexity of inpatient care pathways.
• A gap exists in evaluating LLMs for real-world inpatient decision support and operational efficiency.

Purpose of the Study:

• To address the lack of large-scale datasets for inpatient clinical decision-making.
• To introduce a novel benchmark and framework for evaluating LLMs in inpatient settings.
• To enhance the coordination and efficiency of inpatient care pathways using AI.

Main Methods:

• Developed the Inpatient Pathways Dataset (IPDS) with 51,274 cases across 9 triage departments, 17 disease categories, and 16 treatment options.
• Proposed the Multi-Agent Inpatient Pathways (MAP) framework, featuring specialized triage, diagnosis, and treatment agents coordinated by a chief agent.
• Evaluated MAP against state-of-the-art LLMs for alignment with operational protocols.

Main Results:

• The MAP framework demonstrated superior alignment with operational protocols compared to existing state-of-the-art LLMs.
• MAP effectively coordinated specialized clinical agents for triage, diagnosis, and treatment planning.
• The IPDS benchmark provides a robust dataset for advancing research in this domain.

Conclusions:

• The MAP framework and IPDS benchmark represent a significant advancement for inpatient support systems.
• This approach has the potential to enhance operational efficiency and resource planning in healthcare facilities.
• Further development of AI-driven clinical decision support is crucial for modern healthcare.