Knowledge-Based Interpretation of Multi-Modal Clinical Findings: Evaluating a Local Agentic Bridge Between Worlds

Area of Science:

• Artificial Intelligence in Medicine
• Clinical Decision Support Systems
• Natural Language Processing

Background:

• Clinical practice generates unstructured data (free-text reports, scans) that challenge automated interpretation by knowledge-based clinical decision support (CDS) systems.
• Large language models (LLMs) offer potential for interpreting unstructured clinical data but face hurdles in accuracy, infrastructure, and data privacy.

Purpose of the Study:

• To evaluate the accuracy of various LLM sizes in interpreting complex medical data for CDS.
• To assess LLM performance in calling specific medical logic modules for hepatitis serology interpretation from multi-modal inputs.

Main Methods:

• A novel framework was used to test LLMs of different sizes.
• LLMs were tasked with interpreting hepatitis serology data of varying complexity by calling Arden Syntax Medical Logic Modules.
• Performance was evaluated based on the accuracy of parameter extraction and module invocation.

Main Results:

• Computationally lean LLMs demonstrated high accuracy with low-complexity parameters, suggesting clinical feasibility for private CDS.
• Accuracy significantly decreased when LLMs handled more numerous or complex quantitative parameters.
• GPT-OSS, a lean LLM, showed promising results for specific, low-complexity interpretation tasks.

Conclusions:

• Lean LLMs can achieve high accuracy for specific, low-complexity CDS tasks using unstructured multi-modal data.
• Current LLM performance is limited by parameter complexity and quantity, requiring further development for comprehensive clinical decision support.
• Integrating LLMs with CDS systems holds potential for improving automated interpretation of clinical data, provided accuracy challenges are addressed.