相关实验视频

Updated: Jun 16, 2025

In Vivo Modeling of the Morbid Human Genome using Danio rerio

In Vivo Modeling of the Morbid Human Genome using Danio rerio

Published on: August 24, 2013

评估大型语言模型的医学,外语和自我报告的遗传病症描述

Kendall A Flaharty¹, Ping Hu¹, Suzanna Ledgister Hanchard¹

¹Medical Genomics Unit, National Human Genome Research Institute, National Institutes of Health, 10 Center Dr, Bethesda, MD 20892, USA.

American journal of human genetics

|August 15, 2024

概括

大型语言模型 (LLM) 在医学上表现有前途,但像ChatGPT-4这样的封闭源代码模型在识别遗传条件方面明显优于开源选项. 性能因模型大小和提示符类型而异.

科学领域:

人工智能在医学中的应用
医疗保健的自然语言处理.
基因组医学和诊断基因组医学和诊断

背景情况:

大型语言模型 (LLM) 越来越多地被用于医学应用,包括从临床笔记中生成差异诊断.
关于开源与封闭源LLM的比较性能以及它们在医疗与非专业人士查询中的准确性仍然存在关键问题.
评估识别遗传疾病的LLM能力对于其安全有效地融入临床工作流程至关重要.

研究的目的:

评估各种开源和封闭源LLM在从临床医生和非专业人士提示中识别遗传疾病方面的表现.
将LLM的表现与谷歌搜索和Phenomizer等传统方法进行比较.
调查提示工程技术 (上下文提示,关键字删除) 和数据特征 (教科书与现实世界描述) 对LLM准确性的影响.

主要方法:

评价了7个LLM工具 (Llama-2-chat,Vicuna,Medllama2,Bard/Gemini,Claude,ChatGPT3.5,ChatGPT-4) 和2个非LLM工具 (谷歌搜索,Phenomizer).这些工具包括:
使用教科书般的临床医生问题和63种遗传疾病的非专业翻译.
分析了列表类型和定义类型的上下文提示,关键字删除和真正的个人描述对LLM准确性的影响.

主要成果:

封闭源代码LLM的表现优于开源LLM;ChatGPT-4获得了最高的准确性 (89%-90%).
对于开源的LLM,准确性随着模型大小的增加而增加 (7b: 21%-49%, 13b: 41%-51%, >33b: 54%-68%).
在临床医生和非专业人士对3个LLM和谷歌搜索的提示之间存在显著的绩效差距. 列表类提示提高了性能,而定义类提示没有. 对于大多数LLM来说,稀有术语删除降低了准确性,现实世界的描述产生了低准确性 (最大21%).

结论:

与开源替代品相比,封闭源LLM,特别是ChatGPT-4,在识别遗传疾病方面表现出更好的能力.
模型大小和提示工程策略显著影响LLM在此诊断任务中的性能.
需要进一步的研究来提高LLMs的准确性和可靠性,以便用于现实世界的临床应用,特别是针对患者的特定数据.

关键词:

人工智能的人工智能是人工智能.深度学习是一种深度学习.在上下文中提示提示.大型语言模型机器学习是机器学习.医学遗传学医学遗传学医学基因组学医学基因组学

更多相关视频

Augmenting Large Language Models via Vector Embeddings to Improve Domain-Specific Responsiveness

Augmenting Large Language Models via Vector Embeddings to Improve Domain-Specific Responsiveness

Published on: December 6, 2024

Navigating MARRVEL, a Web-Based Tool that Integrates Human Genomics and Model Organism Genetics Information

Navigating MARRVEL, a Web-Based Tool that Integrates Human Genomics and Model Organism Genetics Information

Published on: August 15, 2019

相关概念视频

Human Genetics

Human Genetics

Human genetics provides a profound framework for understanding the interplay between genetic predispositions and human psychology. At the heart of this discipline lies the study of how genes influence physical traits, behaviors, and susceptibility to diseases. Each person carries a unique genetic code that subtly or significantly shapes their psychological and behavioral landscape.
The complex relationship between genetics and psychology is observable through common biological components such...

Genome-wide Association Studies-GWAS

Genome-wide Association Studies-GWAS

Genome-wide association studies or GWAS are used to identify whether common SNPs are associated with certain diseases. Suppose specific SNPs are more frequently observed in individuals with a particular disease than those without the disease. In that case, those SNPs are said to be associated with the disease. Chi-square analysis is performed to check the probability of the allele likely to be associated with the disease.
GWAS does not require the identification of the target gene involved in...

Genetic Screens

Genetic Screens

Genetic screens are tools used to identify genes and mutations responsible for phenotypes of interest. Genetic screens help identify individuals or a group of people at risk of developing genetic diseases and help them with early intervention, targeted therapy, and reproductive options.
Forward genetic screens
Forward or “classical” genetic screens involve creating random mutations in an organism’s DNA using radiation, mutagens, or insertion of additional bases, which...

Behavioral Genetics and Its Designs

Behavioral Genetics and Its Designs

Behavior genetics explores how genetic inheritance influences human behavior. It focuses on how genes, passed from parents to offspring, contribute to the development of behavioral traits and tendencies. This branch of genetics seeks to understand the complex interplay between inherited genetic factors and environmental influences in shaping our behaviors.
The primary methodologies used in behavior genetics include family studies, twin studies, and adoption studies, each providing unique...

Translation

Translation

Translation is the process of synthesizing proteins from the genetic information carried by messenger RNA (mRNA). Following transcription, it constitutes the final step in the expression of genes. This process is carried out by ribosomes, complexes of protein and specialized RNA molecules. Ribosomes, transfer RNA (tRNA), and other proteins produce a chain of amino acids—the polypeptide—as the end product of translation.
Translation Produces the Building Blocks of Life
Proteins are...

Classification of Illness

Classification of Illness

The meaning of illness is individualized to each person who experiences an alteration in health. In contrast, disease is a medical term indicating a pathological change in the structure and function of the body or mind. It is a condition that has specific symptoms and boundaries.
An illness is a response to a disease in which the person's level of functioning is changed compared with a previous level. The general classification of illness includes acute and chronic.
Acute illness is severe...