FHIR-AgentEval:一个模块化沙箱,用于对临床LLM代理进行基准测试,并对存储增强配置进行评估
Research square
|February 23, 2026
概括
大型语言模型 (LLM) 代理可以使用健康七级快速医疗互操作资源 (HL7 FHIR) 改善医疗数据交换. 在现实的临床工作流程中,具有记忆接地功能的LLM代理提高了任务成功率9.1%.
科学领域:
- 医疗信息学 医疗信息学
- 医疗保健中的人工智能
- 临床工作流的优化 临床工作流优化
背景情况:
- 医疗保健数据交换大量使用HL7 FHIR,但其复杂性给临床工作流程带来了挑战.
- 大型语言模型 (LLM) 代理提供了自然语言对FHIR翻译的潜力,但它们的可靠性需要验证.
研究的目的:
- 引入FHIR-AgentEval,这是一个评估框架,用于在医疗保健环境中对LLM代理进行基准评估.
- 评估LLM代理配置,特别是内存和访问FHIR规范对性能的影响.
主要方法:
- 开发了FHIR-AgentEval,一个沙箱,包含43个任务,用于预约管理和基因测试工作流程.
- 在五个LLM代理配置,不同的内存训练和FHIR规范访问上执行了废除研究.
- 验证了代理响应和FHIR服务器状态的确定性.
主要成果:
- 在实验环境中,LLM代理记忆系统持续提高了任务成功率.
- 记忆接地减少了战略失败,包括错误的工具选择和资源类型混.
- 与基线相比,最好的内存配置在保留任务上取得了9.1%的成功改善.
结论:
- 特别是在内存接地的情况下,LLM代理显示出改善HL7 FHIR互操作性的前景.
- FHIR-AgentEval为在临床应用中评估和推进LLM代理提供了一个强大的平台.
- 这项研究表明,向更可靠的临床部署LLM代理医疗保健数据交换的可行途径.
更多相关视频
相关概念视频
SBAR II: Application of SBAR
6.2K
SBAR is an effective communication tool used by healthcare professionals to communicate patient information accurately. SBAR stands for Situation, Background, Assessment, and Recommendation. For a better understanding, an example is given below.
SBAR Report from a Nurse to a Health Care Provider
S: "Hello, Dr. Smith. This is Jane, RN, from the Med Surg unit. I am calling to tell you about Ms. White in Room 210, who is experiencing increased pain and redness at her incision site. Her recent...
SBAR Report from a Nurse to a Health Care Provider
S: "Hello, Dr. Smith. This is Jane, RN, from the Med Surg unit. I am calling to tell you about Ms. White in Room 210, who is experiencing increased pain and redness at her incision site. Her recent...
6.2K
System of Memory
7.6K
Memory is categorized into three major systems: sensory memory, short-term memory (STM), and long-term memory (LTM). These systems differ in their capacity and the duration for which they can hold information. Sensory memory captures raw sensory input from the environment, holding it for just a few seconds or less. For example, on hearing a brief, loud sound, like a car horn honking, the sound seems to linger in the mind for a moment even after it stops. This is an instance of sensory memory...
7.6K
Impression Management Techniques IV: Altercasting
196
Altercasting is a strategic communication technique in which an individual imposes a specific identity or social role onto another person to influence their behavior and shape the interaction. By presuming a role—such as “responsible leader” or “patient person”—altercasting encourages the target to conform to that identity, often aligning their behavior with the expectations associated with the role. The power of this tactic lies in its subtlety; once a role...
196
Understanding Memory
1.6K
Memory is the retention of information or experiences over time, facilitated through three main processes: encoding, storage, and retrieval. Encoding is the process of inputting information into the memory system. For instance, when listening to a lecture, watching a play, reading a book, or having a conversation, the brain is actively encoding information. This initial stage involves transforming sensory input into a form that can be processed and stored by the brain. Various factors, such as...
1.6K
Masking and Demasking Agents
3.7K
EDTA titrations may necessitate masking and demasking agents to temporarily protect a particular metal ion in a mixture from the EDTA reaction. These agents facilitate the sequential analysis of the metal ions by forming stable complexes with some—but not all—metal ions during certain steps.
There are many masking agents, such as cyanide, fluoride, triethanolamine, thiourea, and 2,3-bis(sulfanyl)propan-1-ol (formerly 2,3-dimercapto-1-propanol), with the masking agent chosen based on...
There are many masking agents, such as cyanide, fluoride, triethanolamine, thiourea, and 2,3-bis(sulfanyl)propan-1-ol (formerly 2,3-dimercapto-1-propanol), with the masking agent chosen based on...
3.7K
Role of Cerebellum and Prefrontal Cortex in Memory
1.3K
The cerebellum, while traditionally associated with motor control, also plays a crucial role in memory, particularly in procedural memory, which involves learning motor tasks that become automatic through repetition. For example, studies have shown that when the cerebellum is damaged, individuals or animals lose the ability to learn conditioned motor responses, such as the conditioned eye-blink response in classical conditioning experiments with rabbits. This study demonstrates the...
1.3K


