Knowledge-Based Interpretation of Multi-Modal Clinical Findings: Evaluating a Local Agentic Bridge Between Worlds

Contemporary clinical practice still produces unstructured data like free-text reports or scans, hindering automated interpretation by knowledge-based clinical decision support (CDS) systems that rely on structured data. Large language models (LLMs) show potential for interpreting such findings but face challenges in accuracy, infrastructure demands, and data privacy. Integrating LLMs with modular knowledge-based CDS systems could provide validated interpretations of such findings, but models need to call CDS modules with perfectly accurate parameters. The accuracy of multiple size classes of LLMs calling Arden Syntax Medical Logic Modules for hepatitis serology interpretation of varying complexity from unstructured multi-modal inputs is tested using a novel framework. Computationally lean LLMs like GPT-OSS were found to handle a small amount of low-complexity parameters with high accuracy, approaching clinical feasibility for private and reliable CDS interpretation of multi-modal data. Accuracy decreased sharply for tools involving more numerous or complex quantitative parameters.