关于临床诊断中大型语言模型的局限性
Justin T Reese1, Daniel Danis2, J Harry Caufield1
1Division of Environmental Genomics and Systems Biology, Lawrence Berkeley National Laboratory, Berkeley, CA, 94720, USA.
medRxiv : the preprint server for health sciences
|July 28, 2023
概括
使用像GPT-4这样的大型语言模型 (LLM) 来从临床文本中生成诊断提示显示出适度的性能. 从结构化数据中编程生成的提示提示表现不佳的叙事文本,突出了目前临床使用的局限性.
科学领域:
- 人工智能在医学中的应用
- 临床决策支持系统 临床决策支持系统
背景情况:
- 像GPT-4这样的大型语言模型 (LLM) 具有使用案例报告进行差异诊断的潜力.
- 电子健康记录 (EHR) 缺乏病例报告的叙事细节,并面临隐私限制.
- 开发保护隐私的方法,利用临床数据与LLM是至关重要的.
研究的目的:
- 测试一种分析临床文本以提取本体学术语的方法,并通过程序生成LLMs的无隐私提示.
- 用结构化临床数据评估基于LLM的差异诊断的性能.
主要方法:
- 从75个已发表的案例报告中调查了提示生成方法.
- 从叙述中提取结构化术语 (表型异常,并发症,治疗,实验室测试).
- 从提取的结构化数据编程创建提示.
主要成果:
- 业绩不佳,正确诊断仅在5.3-17.6%的病例中排名第一.
- 来自结构化数据的提示比原始叙事文本表现更差.
- 在不同版本中,GPT-4的性能差异很大.
结论:
- 提示格式的敏感性和LLM版本的不稳定性限制了现有的GPT-4在现实世界临床诊断中的使用.
- 需要进行进一步的研究,以优化从临床数据中快速创建差异诊断.
更多相关视频
相关概念视频
Language and Cognition
375
Language serves as a bridge between ideas and communication, influencing how individuals perceive and interact with the world. Psychologists have long debated whether language shapes thought or vice versa. This discussion gained grip with Edward Sapir and Benjamin Lee Whorf in the 1940s, who proposed that language determines thought, a concept known as linguistic determinism. They suggested that the vocabulary and structure of a language influence how its speakers think and perceive reality.
375
Documentation of Nursing Diagnosis
1.3K
The nurse documents nursing diagnoses and enters them into the patient record. The identified patient's nursing diagnosis is either written out with a plan of care or entered into the electronic health record.
In some settings, data-driven computerized decision support systems are in place, allowing for more accurate nursing diagnoses. The database within one of these systems includes diagnostic labels defining characteristics, activities, and indicators for nursing. A nurse enters...
In some settings, data-driven computerized decision support systems are in place, allowing for more accurate nursing diagnoses. The database within one of these systems includes diagnostic labels defining characteristics, activities, and indicators for nursing. A nurse enters...
1.3K
Classification of Illness
7.6K
The meaning of illness is individualized to each person who experiences an alteration in health. In contrast, disease is a medical term indicating a pathological change in the structure and function of the body or mind. It is a condition that has specific symptoms and boundaries.
An illness is a response to a disease in which the person's level of functioning is changed compared with a previous level. The general classification of illness includes acute and chronic.
Acute illness is severe...
An illness is a response to a disease in which the person's level of functioning is changed compared with a previous level. The general classification of illness includes acute and chronic.
Acute illness is severe...
7.6K
Diagnostic and Statistical Manual of Mental Disorders (DSM)
95
The Diagnostic and Statistical Manual of Mental Disorders (DSM) serves as the primary classification system for mental health disorders, providing standardized diagnostic criteria for clinicians and researchers. First published by the American Psychiatric Association (APA) in 1952, the DSM has undergone several revisions to reflect evolving psychiatric understanding. The fifth edition, DSM-5, released in 2013, introduced key updates that expanded diagnostic categories and modified diagnostic...
95
Methods of Documentation VI: Case Management Model
598
The case management model is a multidisciplinary approach that involves healthcare professionals from diverse disciplines, such as physicians, nurses, therapists, social workers, and pharmacists, working collaboratively to address the various needs of patients. Each healthcare professional brings unique expertise and perspectives, contributing to a more comprehensive understanding of the patient's condition and tailoring treatment plans accordingly.
For example, a patient with a chronic...
For example, a patient with a chronic...
598
Formulating and Validating Nursing Diagnosis I
2.7K
A nursing diagnosis is written when the nurse recognizes a cluster of essential patient data indicating health problems treated with independent nursing interventions. The standardized terminologies of a nursing diagnosis help nurses identify and treat patients' problems. Every electronic health record that uses nursing diagnosis must employ standard diagnostic terminology. Developing an efficient, individualized care plan begins with accurate nursing diagnoses.
There are thirteen domains...
There are thirteen domains...
2.7K


