分析大语言模型与人类表现对遗传学问题的分析
Dat Duong1, Benjamin D Solomon2
1Medical Genomics Unit, Medical Genetics Branch, National Human Genome Research Institute, Bethesda, MD, USA.
European journal of human genetics : EJHG
|May 28, 2023
概括
像ChatGPT这样的大型语言模型 (LLM) 在人类遗传学问题上显示出与人类相比的准确性. 然而,LLM表现出不一致性,并需要进一步开发高风险的应用程序.
科学领域:
- 生物医学信息学是生物医学信息学.
- 医学中的人工智能
- 人类遗传学教育 人类遗传学教育
背景情况:
- 大型语言模型 (LLM) 正在获得潜在的生物医学应用的关注.
- 评估LLM在人体遗传学等专业领域的表现至关重要.
研究的目的:
- 为了比较ChatGPT与人类受访者在人类遗传学多选题中的表现.
- 在生物医学背景下评估LLM响应的准确性和一致性.
主要方法:
- 进行了一组85个关于人类遗传学的多选题.
- 通过将ChatGPT的答案与13,642个人类答案进行比较来评估性能.
- 分析了准确率和响应的一致性.
主要成果:
- 聊天GPT获得了68.2%的准确率,与人类受访者的66.6%准确率 (p=0.8327) 相比.
- 聊天GPT和人类都在记忆问题上表现更好,而不是批判性思维问题 (p<0.0001).
- 16%的时间,ChatGPT为同一个问题提供了不同的答案,为正确和不正确的答案提供了可信的解释.
结论:
- 聊天GPT在人类遗传学知识回忆方面表现出令人印象深刻的能力,但具有显著的局限性.
- 响应的不一致性和批判性思维的缺陷突出了需要改善临床使用的领域.
- 需要进一步开发,以确保LLMs在高风险的生物医学场景中的可靠性和安全性.
相关概念视频
Evolutionary Relationships through Genome Comparisons
5.9K
Genome comparison is one of the excellent ways to interpret the evolutionary relationships between organisms. The basic principle of genome comparison is that if two species share a common feature, it is likely encoded by the DNA sequence conserved between both species. The advent of genome sequencing technologies in the late 20th century enabled scientists to understand the concept of conservation of domains between species and helped them to deduce evolutionary relationships across diverse...
5.9K
Human Genetics
629
Human genetics provides a profound framework for understanding the interplay between genetic predispositions and human psychology. At the heart of this discipline lies the study of how genes influence physical traits, behaviors, and susceptibility to diseases. Each person carries a unique genetic code that subtly or significantly shapes their psychological and behavioral landscape.
The complex relationship between genetics and psychology is observable through common biological components such...
The complex relationship between genetics and psychology is observable through common biological components such...
629
Behavioral Genetics and Its Designs
430
Behavior genetics explores how genetic inheritance influences human behavior. It focuses on how genes, passed from parents to offspring, contribute to the development of behavioral traits and tendencies. This branch of genetics seeks to understand the complex interplay between inherited genetic factors and environmental influences in shaping our behaviors.
The primary methodologies used in behavior genetics include family studies, twin studies, and adoption studies, each providing unique...
The primary methodologies used in behavior genetics include family studies, twin studies, and adoption studies, each providing unique...
430
Comparing Copy Number Variations and SNPs
17.8K
Sequencing of the human genome has opened up several best-kept secrets of the genome. Scientists have identified thousands of genome variations that exist within a population. These variations can be a single nucleotide or a larger chromosomal variation.
Copy number variations or CNVs are the structural variations that cover more than 1kb of DNA sequence. The single nucleotide polymorphism (SNP), on the other hand, is a single nucleotide change or a point mutation that is found in more than 1%...
Copy number variations or CNVs are the structural variations that cover more than 1kb of DNA sequence. The single nucleotide polymorphism (SNP), on the other hand, is a single nucleotide change or a point mutation that is found in more than 1%...
17.8K
Genetic Lingo
103.3K
Overview
103.3K
Incomplete Dominance
22.9K
Gregor Mendel's work (1822 - 1884) was primarily focused on pea plants. Through his initial experiments, he determined that every gene in a diploid cell has two variants called alleles inherited from each parent. He suggested that amongst these two alleles, one allele is dominant in character and the other recessive. The combination of alleles determines the phenotype of a gene in an organism.
22.9K


