评估生成人工智能工具在理解医学论文方面的能力:定性研究
Seyma Handan Akyon1, Fatih Cagatay Akyon2,3, Ahmet Sefa Camyar4
1Golpazari Family Health Center, Bilecik, Turkey.
JMIR medical informatics
|September 4, 2024
概括
大型语言模型 (LLM) 在理解医学研究论文方面表现不同,其中GPT-3.5-Turbo领先. 本研究评估了使用STROBE检查清单进行医学文献分析的LLM理解.
科学领域:
- 人工智能在医学中的应用
- 自然语言处理自然语言处理.
- 医疗信息学 医疗信息学
背景情况:
- 对临床医生来说,理解医学文献是耗时的.
- 需要有效的工具来帮助医生处理复杂的医疗文件.
- 大型语言模型 (LLM) 为医疗信息处理提供了潜在的解决方案.
研究的目的:
- 批判性地评估和比较各种LLMs的理解能力.
- 评估在理解医学研究论文的LLM准确性和效率.
- 使用STROBE检查清单进行标准化评估框架.
主要方法:
- 在医学论文理解中评估生成性AI工具的方法研究.
- 一个基准管道处理了50篇PubMed医学研究论文.
- 六个LLM (GPT-3.5-Turbo,GPT-4,PaLM 2,Claude v1,Gemini Pro) 与15个STROBE衍生问题的专家基准进行了比较.
主要成果:
- GPT-3.5-Turbo获得了最高的精度 (66.9%),其次是GPT-4-1106 (65.6%) 和PaLM 2 (62.1%).
- 在LLM中观察到具有统计意义的绩效差异 (P<.001).
- 新的LLM版本通常表现优于旧版本,在不同纸张部分的表现有显著差异.
结论:
- 这是第一个使用检索增强生成来评估医学论文理解的LLM的研究.
- 士学位显示出提高医学研究效率和支持基于证据的决策的潜力.
- 需要进一步的研究来解决问题格式的影响,潜在的偏见和LLM技术的快速演变.
关键词:
在这里,我们可以看到AIAIAI.聊天GPT 聊天 在GPT 聊天在 GPT 中,GPT 必须是 GPT.在法学士 (LLM) 课程中.在法律上,LLMs.斯特罗贝斯 (STROBE) 是一个草.加强流行病学观察研究的报告.答案 答案 答案 答案 答案答案 答案 答案 答案 答案 答案人工智能的人工智能是人工智能.了解理解理解理解理解产生性的产生性.医疗保健 医疗保健 医疗保健语言模型语言模型语言模型语言模型大型语言模型.机器学习是机器学习.医学 医学 医学 医学 医学自然语言处理自然语言处理.研究论文研究论文研究论文研究论文响应 响应 响应 响应答案 答案 响应 响应科学研究科学研究更多相关视频
07:50A Metadata Extraction Approach for Clinical Case Reports to Enable Advanced Understanding of Biomedical Concepts
Published on: September 20, 2018
15.9K
09:20Cloud-Based Phrase Mining and Analysis of User-Defined Phrase-Category Association in Biomedical Publications
Published on: February 23, 2019
8.7K
相关概念视频
Non-equilibrium in the Cell
4.3K
An important concept in studying metabolism and energy is that of chemical equilibrium. Most chemical reactions are reversible. They can proceed in both directions, releasing energy into their environment in one direction, and absorbing it from the environment in the other direction. The same is true for the chemical reactions involved in cell metabolism, such as the breaking down and building up of proteins into and from individual amino acids, respectively. Reactants within a closed system...
4.3K
Issues And Trends In Healthcare Delivery System
5.6K
The issues and trends in healthcare delivery are constantly changing. The COVID-19 pandemic is one recent issue that wreaked havoc on healthcare systems, causing a shortage of healthcare workers, high demand for medicines and supplies, and increased medical expenditure due to a lack of insurance. Other issues include rising healthcare costs and care fragmentation.
Cost Containment
Payment for healthcare services has historically promoted adoption of costly and often unnecessary or inefficient...
Cost Containment
Payment for healthcare services has historically promoted adoption of costly and often unnecessary or inefficient...
5.6K
Genomics
36.2K
Genomics is the science of genomes: it is the study of all the genetic material of an organism. In humans, the genome consists of information carried in 23 pairs of chromosomes in the nucleus, as well as mitochondrial DNA. In genomics, both coding and non-coding DNA is sequenced and analyzed. Genomics allows a better understanding of all living things, their evolution, and their diversity. It has a myriad of uses: for example, to build phylogenetic trees, to improve productivity and...
36.2K
Patient-centered Care
2.0K
Patient-centered care involves delivering care beyond inpatient hospitalization. Reflective practice can enhance a patient-centered approach. Reflective practice is a process of reasoning that considers all aspects of the present situation, including practicalities, learning from personal practice, and consideration of patient needs. Patients appreciate care decisions made while considering their input. Involving the patient in their care provides the patient with a sense of contribution rather...
2.0K
Animal Mitochondrial Genetics
7.5K
Among all the organelles in an animal cell, only mitochondria have their own independent genomes. Animal mitochondrial DNA is a double-stranded, closed-circular molecule with around 20,000 base pairs. Mitochondrial DNA is unique in that one of its two strands, the heavy, or H, -strand is guanine rich, whereas the complementary strand is cytosine rich and called the light, or L, -strand. Compared to nuclear DNA, mitochondrial DNA has a very low percentage of non-coding regions and is marked by...
7.5K
Combination Therapies and Personalized Medicine
4.9K
Combining two or more treatment methods increases the life span of cancer patients while reducing damage to vital organs or tissue from the overuse of a single treatment. Combination therapy also targets different cancer-inducing pathways, thus reducing the chances of developing resistance to treatment.
The combination of the drug acetazolamide and sulforaphane is a good example of combination therapy to treat cancer. The cells in the interior of a large tumor often die due to the hypoxic and...
The combination of the drug acetazolamide and sulforaphane is a good example of combination therapy to treat cancer. The cells in the interior of a large tumor often die due to the hypoxic and...
4.9K
