LLMを用いた動脈血液ガス分析の解釈：3群試験における新しい数学的スクラッチパッドと異なるプロンプト手法の比較

Large language models (LLMs) have demonstrated proficiency in various tasks, yet their effectiveness in a clinical decision support system (CDSS) is evolving. One challenge is their limited ability to perform calculations. This study evaluates a novel method of using a custom math scratchpad to perform domain-specific calculations in interpreting Arterial Blood Gases (ABGs). Three methods are compared: zero-shot prompting (Method 1), prompt engineering with Retrieval-Augmented Generation (RAG) (Method 2), and a combined novel math scratchpad, RAG, and prompt engineering (Method 3). The LLM-integrated CDSS achieved an accuracy rate of 86% (43/50) [confidence interval (CI) 74%-93%] across a dataset of 50 ABG results when utilizing Method 3, compared to 78% (39/50) [CI 65%-87%] for Method 2 and 48% (24/50) [CI 35%-61%] for Method 1. The evaluation demonstrates a math scratchpad's utility in ABG interpretation by overcoming LLMs' calculation limitation. Further testing with real-world patient ABG data is needed.