相关实验视频

Updated: May 9, 2025

Augmenting Large Language Models via Vector Embeddings to Improve Domain-Specific Responsiveness

Augmenting Large Language Models via Vector Embeddings to Improve Domain-Specific Responsiveness

Published on: December 6, 2024

专家的专家验证和调整 (EVAL) 框架大语言模型安全在胃肠道学

Mauro Giuffrè¹, Kisung You², Ziteng Pang³

¹Section of Digestive Diseases, Department of Medicine, Yale School of Medicine, New Haven, USA.

NPJ digital medicine

|May 3, 2025

概括

我们开发了EVAL,这是一种用于验证医学查询的大型语言模型 (LLM) 安全性的新方法. EVAL简化了LLM输出分级,提高了关键应用程序的准确性,例如上部胃肠道出血 (UGIB).

科学领域:

人工智能在医学中的应用
自然语言处理自然语言处理.
临床决策支持临床决策支持

背景情况:

大型语言模型 (LLM) 为医学问题解答提供了潜力,但由于输出不准确而带来风险.
在临床环境中,手动对LLM反应进行分级是不切实际的,这阻碍了安全实施.
确保LLM准确性对于高风险的医疗决策至关重要.

研究的目的:

引入EVAL (专家中的专家验证和调整),这是一个旨在简化LLM输出评估的系统.
提高LLM在医疗应用中的安全性和准确性,特别是对于上部胃肠道出血 (UGIB).
为了比较各种LLM配置和评估技术.

主要方法:

在27个配置中评估了多个LLM (GPT,Claude,LLaMA,Mixtral),包括零射击,检索增强生成和监督微调.
实施了EVAL,使用基于相似性的排名与微调ColBERT以及训练在拒绝采样的人类等级响应上的奖励模型.
通过使用三个单独的数据集,评估了与人类表现的调整.

主要成果:

微调的ColBERT显示了与人类表现的最高对齐 (ρ = 0.81-0.91).
在87.9%的案例中,奖励模型准确地复制了人类的分级.
使用EVAL的拒绝采样显著提高了LLM的准确性,总体上提高了8.36%.

结论:

EVAL提供了一个可扩展的解决方案,用于在医疗环境中评估LLM准确性.
开发的系统提高了临床决策支持的LLM安全性和可靠性.
这种方法对于在医疗保健中负责任地部署AI至关重要.

更多相关视频

Foreign Accent and Forensic Speaker Identification in Voice Lineups: The Influence of Acoustic Features Based on Prosody

Foreign Accent and Forensic Speaker Identification in Voice Lineups: The Influence of Acoustic Features Based on Prosody

Published on: September 27, 2024

Objectification of Tongue Diagnosis in Traditional Medicine, Data Analysis, and Study Application

Objectification of Tongue Diagnosis in Traditional Medicine, Data Analysis, and Study Application

Published on: April 14, 2023

相关概念视频

Improving Translational Accuracy

Improving Translational Accuracy

Leaky Scanning

Leaky Scanning

During most eukaryotic translation processes, the small 40S ribosome subunit scans an mRNA from its 5' end until it encounters the first start AUG codon. The large 60S ribosomal subunit then joins the smaller one to initiate protein synthesis. The location of the translation initiation is largely determined by the nucleotides near the start codon as there may be multiple translation initiation sites present on the mRNA. Marilyn Kozak discovered that the sequence RCCAUGG (where R...