相关实验视频

Updated: Aug 4, 2026

A Machine Learning Approach to Design an Efficient Selective Screening of Mild Cognitive Impairment

A Machine Learning Approach to Design an Efficient Selective Screening of Mild Cognitive Impairment

Published on: January 11, 2020

大型语言模型不确定性代理:医疗诊断和治疗的歧视和校准

Thomas Savage^1,2, John Wang^1,3, Robert Gallo^4,5

¹Department of Medicine, Stanford University, Stanford, CA 94304, United States.

Journal of the American Medical Informatics Association : JAMIA

|October 13, 2024

概括

样本一致性 (SC) 有效地估计了医疗任务中的大语言模型 (LLM) 不确定性. 虽然通过句子嵌入的SC显示了高度的歧视,但通过GPT注释的SC为医生提供了更好的校准.

科学领域:

人工智能在医学中的应用
自然语言处理自然语言处理.
临床决策支持临床决策支持

背景情况:

大型语言模型 (LLM) 在传达不确定性方面面临挑战,这阻碍了它们的医学应用.
可靠的不确定性估计对于将LLM整合到患者护理中至关重要.

研究的目的:

评估不确定性代理来量化LLM对诊断和治疗选择的信心.
评估这些代理的区分和校准特性.

主要方法:

在多个LLM (GPT3.5,GPT4,Llama2,Llama3) 中检查了信心诱导 (CE),代币级概率 (TLP) 和样本一致性 (SC).
评估了使用三个开放式患者场景数据集的代理.

主要成果:

与TLP和CE相比,样本一致性 (SC) 显示出更高的歧视性.
通过句子嵌入的SC实现了高差别 (ROC AUC 0.68-0.79),但校准不佳.
通过GPT注释的SC提供了强烈的歧视 (ROC AUC 0.66-0.74) 与准确的校准.
口头表达的信心 (CE) 始终高估了LLM的信心.

结论:

在评估的代理程序中,SC是估计LLM不确定性的最有效方法.
通过GPT注释的SC可以在没有参考案例的情况下提供准确的校准.
通过句子嵌入的SC需要重新校准,使用参考案例进行最佳使用.
在口头表达信心 (CE) 时,LLM表现出一致的过度自信.

关键词:

在法学士 (LLM) 课程中.大型语言模型校准校准的时间信心信心信心信心信心歧视是一种歧视.不确定性是一种不确定性.

更多相关视频

Objectification of Tongue Diagnosis in Traditional Medicine, Data Analysis, and Study Application

Objectification of Tongue Diagnosis in Traditional Medicine, Data Analysis, and Study Application

Published on: April 14, 2023

Augmenting Large Language Models via Vector Embeddings to Improve Domain-Specific Responsiveness

Augmenting Large Language Models via Vector Embeddings to Improve Domain-Specific Responsiveness

Published on: December 6, 2024

相关概念视频

Data Validation

Data Validation

Data validation is an essential part of a comprehensive assessment. Validation is confirming or verifying and opening the door to gathering more assessment data as it clarifies vague or unclear data. The process of checking and verifying the collected information is called data validation. The primary purpose of data validation is to ensure data is as free from error, bias, and misinterpretation as possible.
Nursing assessment guides are generally based on holistic models rather than medical...

Diagnostic and Statistical Manual of Mental Disorders (DSM)

Diagnostic and Statistical Manual of Mental Disorders (DSM)

The Diagnostic and Statistical Manual of Mental Disorders (DSM) serves as the primary classification system for mental health disorders, providing standardized diagnostic criteria for clinicians and researchers. First published by the American Psychiatric Association (APA) in 1952, the DSM has undergone several revisions to reflect evolving psychiatric understanding. The fifth edition, DSM-5, released in 2013, introduced key updates that expanded diagnostic categories and modified diagnostic...