相关实验视频

Updated: May 24, 2025

Augmenting Large Language Models via Vector Embeddings to Improve Domain-Specific Responsiveness

Augmenting Large Language Models via Vector Embeddings to Improve Domain-Specific Responsiveness

Published on: December 6, 2024

评估基础和检索增强的LLM与基于证据的神经学的文档或在线支持

Lars Masanneck¹, Sven G Meuth², Marc Pawlitzki²

¹Department of Neurology, Medical Faculty and University Hospital Düsseldorf, Heinrich Heine University Düsseldorf, Düsseldorf, Germany. lars.masanneck@med.uni-duesseldorf.de.

NPJ digital medicine

|March 4, 2025

概括

大型语言模型 (LLM) 和检索增强生成 (RAG) 系统在管理基于证据的神经信息方面表现出不同的准确性. 虽然RAG可以提高性能,但进一步开发对于安全的临床使用至关重要.

科学领域:

医疗信息学医疗信息学
人工智能在医学中的应用
临床决策支持临床决策支持

背景情况:

管理基于证据的医疗信息是复杂的和不断增长的.
大型语言模型 (LLM) 为信息合成提供了潜在的解决方案.
检索增强生成 (RAG) 旨在通过外部数据提高LLM的准确性.

研究的目的:

根据神经学指南,评估LLMs和RAG系统在回答临床问题的性能.
为了比较文件启用和在线启用RAG系统与基础LLM性能.
确定医疗应用RAG系统的局限性和需要改进的领域.

主要方法:

测试了LLM和RAG系统 (支持文档和在线) 的130个问题,这些问题来自13个神经学指南.
评估准确性和确定类型的错误,包括潜在的有害反应.
将问题分为基于知识和基于案例的类别,用于比较分析.

主要成果:

在不同的LLM和RAG配置中观察到显著的性能变化.
虽然RAG系统的准确性比基础LLM更高,但仍会产生不准确性.
与基于知识的问题相比,RAG系统在基于病例的临床场景中表现不佳.

结论:

目前的RAG增强的LLMs需要大幅改进,以实现可靠的临床整合.
在医疗保健中广泛采用之前,需要提高准确性和安全性措施.
进一步的研究和监管监督对于在临床实践中负责任地实施AI至关重要.

更多相关视频

Multimodal Protocol for Assessing Metacognition and Self-Regulation in Adults with Learning Difficulties

Multimodal Protocol for Assessing Metacognition and Self-Regulation in Adults with Learning Difficulties

Published on: September 27, 2020

Measuring Statistical Learning Across Modalities and Domains in School-Aged Children Via an Online Platform and Neuroimaging Techniques

Measuring Statistical Learning Across Modalities and Domains in School-Aged Children Via an Online Platform and Neuroimaging Techniques

Published on: June 30, 2020

相关概念视频

Long-term Potentiation

Long-term Potentiation

Long-term potentiation, or LTP, is one of the ways by which synaptic plasticity—changes in the strength of chemical synapses—can occur in the brain. LTP is the process of synaptic strengthening that occurs over time between pre- and postsynaptic neuronal connections. The synaptic strengthening of LTP works in opposition to the synaptic weakening of long-term depression (LTD) and together are the main mechanisms that underlie learning and memory.