Challenges and Solutions in Applying Large Language Models to Guideline-Based Management Planning and Automated...

Area of Science:

• Healthcare technology
• Artificial intelligence in medicine
• Clinical informatics

Background:

• Diagnostic errors and administrative burdens, such as medical coding, are significant healthcare challenges.
• Large language models (LLMs) show promise for addressing these issues, but concerns about reliability and clinical safety hinder adoption.

Purpose of the Study:

• To introduce and evaluate two LLM-based frameworks: GARAG for automated evidence-based treatment planning and GAVS for automated medical coding.
• To assess the effectiveness of these frameworks within the Rhazes Clinician platform.

Main Methods:

• GARAG was evaluated on 21 clinical test cases, assessing reference correctness, duplication, formatting, and clinical appropriateness.
• GAVS was evaluated on 958 intensive care admissions, comparing its performance against a direct GPT-4.1 baseline for ICD-10 code prediction.

Main Results:

• GARAG outputs met all evaluation criteria in 98.4% of cases, demonstrating a workflow grounded in evidence.
• GAVS achieved a statistically significant improvement in fine-grained diagnostic coding recall (20.63% vs 17.95%) compared to the baseline LLM.

Conclusions:

• LLM-based frameworks like GARAG and GAVS can enhance clinical decision support and medical coding.
• These frameworks, integrated into Rhazes Clinician, offer a unified interface for physicians, though further validation is needed.