在基于文本的放射学知识评估中,ChatGPT和Bard的比较性能
Nikhil S Patil1, Ryan S Huang2, Christian B van der Pol3
1Michael G. DeGroote School of Medicine, McMaster University, Hamilton, ON, Canada.
概括
与谷歌巴德相比,ChatGPT-4在回答放射学委员会问题方面表现出更高的准确性,突出了AI聊天机器人的医疗教育性能差异. 这两种工具都有局限性,应谨慎使用.
科学领域:
- 医疗教育中的人工智能
- 放射学委员会考试的准备工作 放射学委员会考试的准备工作
- 在医疗保健中的自然语言处理.
背景情况:
- 最近发布的Google Bard,是ChatGPT的竞争对手,需要对其能力进行评估.
- 了解人工智能聊天机器人的比较优缺点对于它们在放射学等专业领域的有效应用至关重要.
- 在医学教育中评估人工智能的表现可以指导这些工具的开发和部署.
研究的目的:
- 为了比较ChatGPT-4和Google Bard在准确回答放射学委员会考试实践问题的表现.
- 分析这两个领先的人工智能聊天机器人的相对准确性,响应长度和响应时间.
- 评估ChatGPT-4和Bard之间的子专业特定性能差异.
主要方法:
- 基于文本的问题来源于美国放射学学院的诊断放射学培训 (DXIT) 考试 (2017-2021年).
- 聊天GPT-4和巴德被问到这些问题,他们的答案被系统地记录下来.
- 对比分析包括准确率,响应长度的字符数和响应生成所需的时间.
主要成果:
- 在318个问题 (P < .0001) 中,ChatGPT-4的准确率 (87.11%) 比Bard (70.44%) 高得多.
- 聊天GPT-4提供更短的响应 (935.28字符) 比巴德 (1437.52字符) (P < .0001),但需要更长的响应时间 (26.79秒对比7.55秒) (P < .0001).
- 在神经放射学,一般物理,核医学,儿科放射学和超声波方面,ChatGPT-4的表现优于Bard,在其他子专业方面没有显著差异.
结论:
- 与谷歌Bard相比,ChatGPT-4在放射学方面表现出卓越的知识.
- 虽然这两种聊天机器人都具有一定的放射学知识,但用户必须意识到它们的局限性和潜在的不准确性.
- 两种人工智能模型都偶尔提供了有缺陷的解释,或者未能充分解决问题的教育背景.
更多相关视频
05:47Evidence-based Knowledge Synthesis and Hypothesis Validation: Navigating Biomedical Knowledge Bases via Explainable AI and Agentic Systems
Published on: June 13, 2025
287
06:09P300-Based Brain-Computer Interface Speller Performance Estimation with Classifier-Based Latency Estimation
Published on: September 8, 2023
614
相关概念视频
01:29Positron Emission Tomography
4.3K
Positron emission tomography (PET) is a medical imaging technique involving radiopharmaceuticals — substances that emit short-lived radiation. Although the first PET scanner was introduced in 1961, it took 15 more years before radiopharmaceuticals were combined with the technique and revolutionized its potential.
One of the main requirements of a PET scan is a positron-emitting radioisotope, which is produced in a cyclotron and then attached to a substance used by the part of the body...
One of the main requirements of a PET scan is a positron-emitting radioisotope, which is produced in a cyclotron and then attached to a substance used by the part of the body...
4.3K
01:15Myasthenia Gravis: Diagnostic Tests
1.0K
Myasthenia gravis is an autoimmune condition affecting neuromuscular transmission, causing generalized weakness in skeletal muscles. Initial diagnoses rely on patients' signs, symptoms, and medical history. The challenge lies in distinguishing myasthenia from other muscular dystrophies. An important diagnostic feature is the significant improvement of symptoms after administering anticholinesterase inhibitors.
The edrophonium test is a diagnostic tool for myasthenia gravis. It involves...
The edrophonium test is a diagnostic tool for myasthenia gravis. It involves...
1.0K