模拟评估大型语言模型逐步诊断推理与现实世界胸痛遭遇和贝叶斯网络的模拟评估
Conrad W Safranek1,2, Vimig Socrates1,3, Donald Wright1,2
1Department of Biomedical Informatics and Data Science, School of Medicine, Yale University, New Haven, CT, USA.
BMC medical informatics and decision making
|February 24, 2026
概括
像GPT-4o这样的大型语言模型显示诊断偏差,在模拟的急诊室病例中过度预测罕见疾病. 他们的信息搜索策略也不同于最佳贝叶斯网络和医生实践.
科学领域:
- 人工智能在医学中的应用
- 临床决策支持系统 临床决策支持系统
- 医学诊断 医学诊断 医学诊断
背景情况:
- 在临床环境中评估大型语言模型 (LLM) 具有挑战性,因为它依赖于静态数据.
- 需要真实世界的模拟来评估LLM的动态诊断能力.
- 作为临床诊断辅助剂,GPT-4o的性能需要与既有方法进行基准测试.
研究的目的:
- 在模拟的急诊室环境中对GPT-4o的诊断准确性和信息搜索行为进行基准测试.
- 为了比较GPT-4o的表现与贝叶斯网络衍生的最佳政策和观察到的医生实践.
- 评估GPT-4o的诊断偏差和对临床决策的潜在影响.
主要方法:
- 通过使用真实世界的临床数据,对500个模拟的急诊室胸痛遭遇进行了评估.
- 用贝叶斯网络 (BN) 创建了一个强大的模拟环境,并推导出最佳查询路径.
- 在不同的提示策略下分析了GPT-4o的顺序信息请求和诊断决策.
主要成果:
- GPT-4o系统地预测过多的罕见疾病与基线提示 (敏感度79.3%,特异性45.2%).
- 调整提示提高了特异性,但显著降低了灵敏度,表明了诊断偏差.
- GPT-4o的信息搜索与BN最佳和医生实践不同,要求更少的生命体/实验室和更多的成像.
结论:
- GPT-4o表现出对罕见疾病的诊断偏见,并与概率模型和医生行为分歧.
- 这些差异可能导致不必要的过分分类和资源利用.
- 将LLM整合到概率框架中,并将其校准到现实的流行率中,对于有效的临床决策支持至关重要.
相关概念视频
Formulating and Validating Nursing Diagnosis II
4.2K
Nursing diagnoses represent a problem validated by major defining characteristics. There are four categories of nursing diagnoses: problem-focused, risk, health promotion or wellness, and syndrome. The anatomy of a nursing diagnosis includes three components: problem statement or diagnostic label, defining characteristics, and related factors.
Risk nursing diagnoses represent clinical judgments of an individual, family, or community more vulnerable to developing the health problem than others...
Risk nursing diagnoses represent clinical judgments of an individual, family, or community more vulnerable to developing the health problem than others...
4.2K
Formulating and Validating Nursing Diagnosis I
4.2K
A nursing diagnosis is written when the nurse recognizes a cluster of essential patient data indicating health problems treated with independent nursing interventions. The standardized terminologies of a nursing diagnosis help nurses identify and treat patients' problems. Every electronic health record that uses nursing diagnosis must employ standard diagnostic terminology. Developing an efficient, individualized care plan begins with accurate nursing diagnoses.
There are thirteen domains...
There are thirteen domains...
4.2K
Documentation of Nursing Diagnosis
1.9K
The nurse documents nursing diagnoses and enters them into the patient record. The identified patient's nursing diagnosis is either written out with a plan of care or entered into the electronic health record.
In some settings, data-driven computerized decision support systems are in place, allowing for more accurate nursing diagnoses. The database within one of these systems includes diagnostic labels defining characteristics, activities, and indicators for nursing. A nurse enters...
In some settings, data-driven computerized decision support systems are in place, allowing for more accurate nursing diagnoses. The database within one of these systems includes diagnostic labels defining characteristics, activities, and indicators for nursing. A nurse enters...
1.9K
Angina III: Clinical Manifestations and Assessment
301
Angina manifests as chest pain, tightness, or squeezing discomfort typically located behind the breastbone. It can radiate to the neck, jaw, shoulders, and inner aspects of the upper arms, most commonly the left arm. Patients may experience shortness of breath, fatigue, profuse sweating, dizziness, indigestion, heartburn, palpitations, anxiety, and vomiting as accompanying symptoms. This pain often lasts a few minutes and is triggered by physical exertion, emotional stress, heavy meals, or cold...
301
Classification of Illness
9.0K
The meaning of illness is individualized to each person who experiences an alteration in health. In contrast, disease is a medical term indicating a pathological change in the structure and function of the body or mind. It is a condition that has specific symptoms and boundaries.
An illness is a response to a disease in which the person's level of functioning is changed compared with a previous level. The general classification of illness includes acute and chronic.
Acute illness is severe...
An illness is a response to a disease in which the person's level of functioning is changed compared with a previous level. The general classification of illness includes acute and chronic.
Acute illness is severe...
9.0K

