相关实验视频

Updated: Jun 5, 2025

Augmenting Large Language Models via Vector Embeddings to Improve Domain-Specific Responsiveness

Augmenting Large Language Models via Vector Embeddings to Improve Domain-Specific Responsiveness

Published on: December 6, 2024

关于大型语言模型遵守指南的自主医疗评估

Dennis Fast¹, Lisa C Adams², Felix Busch³

¹DATEXIS, Berliner Hochschule für Technik (BHT), Berlin, Germany.

NPJ digital medicine

|December 12, 2024

概括

自主医疗评估指南遵守 (AMEGA) 基准测试医疗指南的大型语言模型. 在性能方面,GPT-4领先,表现优于最近的医学毕业生.

科学领域:

人工智能在医学中的应用
医疗信息学医疗信息学
临床决策支持系统临床决策支持系统

背景情况:

大型语言模型 (LLM) 在医疗保健中表现有前途,但需要严格评估临床指南的遵守.
现有的基准可能容易受到LLM对培训数据的记忆的影响.
标准化评估对于验证医疗应用中的AI工具至关重要.

研究的目的:

引入自主医疗评估指南遵守 (AMEGA) 基准来评估LLM遵守医疗指南的标准.
提供一个强大的评估框架,涵盖医疗推理,诊断和治疗计划.
促进对人工智能辅助临床决策的研究.

主要方法:

开发了AMEGA,这是一个具有13个专业的20个诊断场景的基准.
创建了135个开放式问题和1337个加权的评分元素,反映了临床相互作用.
通过使用AMEGA框架评估了17个LLM,结合了新内容以防止记忆.

主要成果:

在AMEGA基准指标上,GPT-4获得了最高的得分 (41.9/50).
拉玛-3 70B和巫师LM-2-8x22B也表现出了强的表现.
法学士的成绩明显超过了最近的医学毕业生 (25.8/50).

结论:

AMEGA基准有效地评估了LLM对医疗指南的遵守.
最先进的LLM在医学推理和指南应用方面展示了先进的能力.
AMEGA的开源性质支持进一步开发用于临床决策支持的AI工具.

更多相关视频

Objectification of Tongue Diagnosis in Traditional Medicine, Data Analysis, and Study Application

Objectification of Tongue Diagnosis in Traditional Medicine, Data Analysis, and Study Application

Published on: April 14, 2023

Foreign Accent and Forensic Speaker Identification in Voice Lineups: The Influence of Acoustic Features Based on Prosody

Foreign Accent and Forensic Speaker Identification in Voice Lineups: The Influence of Acoustic Features Based on Prosody

Published on: September 27, 2024

相关概念视频

Methods of Documentation VI: Case Management Model

Methods of Documentation VI: Case Management Model

The case management model is a multidisciplinary approach that involves healthcare professionals from diverse disciplines, such as physicians, nurses, therapists, social workers, and pharmacists, working collaboratively to address the various needs of patients. Each healthcare professional brings unique expertise and perspectives, contributing to a more comprehensive understanding of the patient's condition and tailoring treatment plans accordingly.
For example, a patient with a chronic...

Mechanistic Models: Compartment Models in Algorithms for Numerical Problem Solving

Mechanistic Models: Compartment Models in Algorithms for Numerical Problem Solving

Mechanistic models play a crucial role in algorithms for numerical problem-solving, particularly in nonlinear mixed effects modeling (NMEM). These models aim to minimize specific objective functions by evaluating various parameter estimates, leading to the development of systematic algorithms. In some cases, linearization techniques approximate the model using linear equations.
In individual population analyses, different algorithms are employed, such as Cauchy's method, which uses a...