评估ChatGPT,谷歌Bard和微软Bing在解决放射学病例标签方面的能力
Pradosh Kumar Sarangi1, Ravi Kant Narayan2, Sudipta Mohakud3
1Department of Radiodiagnosis, All India Institute of Medical Sciences, Deoghar, Jharkhand, India.
The Indian journal of radiology & imaging
|March 29, 2024
概括
人类放射科医生在解释放射病例的细节方面显著超过了像ChatGPT,Google Bard和Bing这样的AI模型. 尽管有了人工智能协议,但人类的专业知识仍然优越于准确的医学图像解释和诊断.
科学领域:
- 医疗成像中的人工智能
- 辐射学教育和评估
- 在医疗保健中的自然语言处理.
背景情况:
- 准确的医学图像解释对于放射学诊断和患者护理至关重要.
- 人工智能和NLP的进步促使人们对人工智能在协助放射科医生的作用进行调查.
- 关于人工智能在放射学病例解释与人类专家中的表现的研究有限.
研究的目的:
- 为了评估ChatGPT,谷歌Bard和Bing在放射病例简介上的表现.
- 将人工智能模型的反应与两名放射科住院医生的反应进行比较.
- 评估AI在解决皇家放射科医生学院2A (FRCR2A) 风格问题的能力.
主要方法:
- 使用了120个基于放射病例简报 (FRCR2A风格) 的多项选择题.
- 人工智能模型 (ChatGPT,Bard,Bing) 和两名放射科医生回答了这些问题.
- 人工智能答案与答案键相比较,解释由放射科医生评分;设置了60%的通过分数.
主要成果:
- 居民 (63.33%,57.5%) 的表现优于人工智能模型 (Bard 44.17%,Bing 53.33%,ChatGPT 45%);只有一个居民通过.
- 响应模式有显著差异 (p=0.0117);人工智能模型显示高达成一致 (ICC=0.628),而居民显示低达成一致 (Kappa=-0.376).
- 人工智能对答案的解释只有44.72%的准确率.
结论:
- 与评估的AI模型相比,人类放射科医生表现出更高的准确性和理解力.
- 所有测试的AI模型都未能达到FRCR2A考试的通过分数.
- 人工智能模型表现出一致的反应,与居民不同,表明AI在标准化评估中的潜力,但突出了当前细微解释的局限性.
相关概念视频
Radiological Investigation II: MRI and Ventilation Perfusion Scan
119
Description
Magnetic Resonance Imaging (MRI) and Ventilation Perfusion Scans are two radiological investigations that offer detailed diagnostic images of the body, particularly lung structures.
MRI
MRI uses magnetic fields and radiofrequency signals to distinguish between normal and abnormal tissues. This technology provides a more detailed diagnostic image than CT scans, enabling it to characterize pulmonary nodules, stage bronchogenic carcinoma, and evaluate inflammatory activity in...
Magnetic Resonance Imaging (MRI) and Ventilation Perfusion Scans are two radiological investigations that offer detailed diagnostic images of the body, particularly lung structures.
MRI
MRI uses magnetic fields and radiofrequency signals to distinguish between normal and abnormal tissues. This technology provides a more detailed diagnostic image than CT scans, enabling it to characterize pulmonary nodules, stage bronchogenic carcinoma, and evaluate inflammatory activity in...
119
Positron Emission Tomography
4.2K
Positron emission tomography (PET) is a medical imaging technique involving radiopharmaceuticals — substances that emit short-lived radiation. Although the first PET scanner was introduced in 1961, it took 15 more years before radiopharmaceuticals were combined with the technique and revolutionized its potential.
One of the main requirements of a PET scan is a positron-emitting radioisotope, which is produced in a cyclotron and then attached to a substance used by the part of the body...
One of the main requirements of a PET scan is a positron-emitting radioisotope, which is produced in a cyclotron and then attached to a substance used by the part of the body...
4.2K
Radiological Investigation I: X-ray and CT
237
Radiological investigations, including X-rays and computed tomography (CT) scans, are critical for diagnosing and evaluating various medical conditions. These imaging techniques provide valuable insights into the body's internal structures, aiding in the detection of abnormalities, assessment of disease progression, and development of treatment strategies. This article delves into two primary radiological investigations, chest X-rays and CT scans, outlining their purpose, procedures, and...
237
Radiological Investigation III: Pulmonary Angiogram and PET Scan
91
Radiological investigations are paramount in the diagnosis and management of various pulmonary diseases. Two essential investigations are the Pulmonary Angiogram and the Positron Emission Tomography (PET) Scan.
Pulmonary Angiogram
A Pulmonary Angiogram is an invasive procedure involving injecting a contrast medium through a catheter threaded into the pulmonary artery or the right side of the heart to visualize the pulmonary vasculature. Computed Tomography (CT) scans have mainly replaced this...
Pulmonary Angiogram
A Pulmonary Angiogram is an invasive procedure involving injecting a contrast medium through a catheter threaded into the pulmonary artery or the right side of the heart to visualize the pulmonary vasculature. Computed Tomography (CT) scans have mainly replaced this...
91


