对表型概念识别GPT模型的评估
Tudor Groza1,2,3,4, Harry Caufield5, Dylan Gration6
1Rare Care Centre, Perth Children's Hospital, 15 Hospital Avenue, Nedlands, WA, 6009, Australia. tudor.groza@health.wa.gov.au.
BMC medical informatics and decision making
|January 31, 2024
概括
大型语言模型 (LLM) 对临床表型和表型注释有希望,在使用上下文学习时,GPT-4的表现优于现有的工具. 然而,由于LLM的变化和成本,仍然存在挑战.
科学领域:
- 医疗信息学 医疗信息学
- 计算生物学 计算生物学
- 自然语言处理自然语言处理.
背景情况:
- 临床深度表型和表型注释对于罕见疾病诊断和知识建设至关重要.
- 这些过程传统上使用本体学概念和机器学习来进行表型识别.
- 大型语言模型 (LLM) 越来越多地用于自然语言处理 (NLP) 任务.
研究的目的:
- 评估最新的生成预训练变压器 (GPT) 模型的性能,特别是ChatGPT (GPT-3.5-turbo和GPT-4.0),用于临床表型和表型注释.
- 为了比较LLM的表现与既定的黄金标准公司和现有工具.
主要方法:
- 在GPT-3.5-turbo和GPT-4.0模型中使用了不同特性的七个提示符.
- 评估了两种黄金标准体的性能:出版摘要和临床观察.
- 采用语境学习作为一个关键的实验参数.
主要成果:
- 最好的表现,使用GPT-4.0与上下文学习,在文档级F1得分达到0.58的摘要和0.75临床观察.
- 0.7的F1评分超过了当前最好的工具.
- 在没有上下文学习的情况下,表现显著下降.
结论:
- 当任务被限制在已知的本体学子集时,GPT-4.0展示了表型识别的最先进性能.
- 有希望的结果受到LLM非确定性,高成本和缺乏运行对运行的一致性的影响,这给临床应用带来了挑战.
相关概念视频
01:24Concepts and Prototypes
143
The human nervous system handles vast amounts of information by translating sensory stimuli into neural impulses, which the brain processes, creating thoughts expressed through language or stored as memories. The brain also synthesizes information from emotions and memories, which significantly influence thoughts and behaviors. This intricate process creates a comprehensive mental picture.
The brain organizes this information using concepts, which are mental categories grouping linguistic data,...
The brain organizes this information using concepts, which are mental categories grouping linguistic data,...
143
02:27Background and Environment Affect Phenotype
6.5K
Although the genetic makeup of an organism plays a major role in determining the phenotype, there are also several environmental factors, such as temperature, oxygen availability, presence of mutagens, that can alter an organism’s phenotype.
An example of how genetic background affects phenotype can be seen in horses. The Extension gene in horses is responsible for their coat color. A wild-type gene (EE) produces black pigment in the coat, while a mutant gene (ee) produces red pigment. A...
An example of how genetic background affects phenotype can be seen in horses. The Extension gene in horses is responsible for their coat color. A wild-type gene (EE) produces black pigment in the coat, while a mutant gene (ee) produces red pigment. A...
6.5K
01:18Polygenic Traits
65.8K
When more than one gene is responsible for a given phenotype, the trait is considered polygenic. Human height is a polygenic trait. Studies have uncovered hundreds of loci that influence height, and there are believed to be many more. Due to the high number of genes involved, as well as environmental and nutritional factors, height varies significantly within a given population. The distribution of height forms a bell-shaped curve, with relatively few individuals in the population at the...
65.8K
01:11Genome-wide Association Studies-GWAS
13.4K
Genome-wide association studies or GWAS are used to identify whether common SNPs are associated with certain diseases. Suppose specific SNPs are more frequently observed in individuals with a particular disease than those without the disease. In that case, those SNPs are said to be associated with the disease. Chi-square analysis is performed to check the probability of the allele likely to be associated with the disease.
GWAS does not require the identification of the target gene involved in...
GWAS does not require the identification of the target gene involved in...
13.4K
01:35Pedigree Analysis
84.3K
Overview
84.3K