评估GPT大型语言模型在RSNA2023上的表现案例当天的问题
Pritam Mukherjee1, Benjamin Hou1, Abhinav Suri1
1From the Department of Radiology and Imaging Sciences, Imaging Biomarkers and Computer-Aided Diagnosis Laboratory, NIH Clinical Center, 10 Center Dr, Bldg 10, Rm 1C224D, Bethesda, MD 20892-1182 (P.M., B.H., A.S., Y.Z., R.M.S.); Walter Reed National Military Medical Center, Bethesda, Md (C.P., N.L., O.S.); Radiologic Associates of Middletown, Middletown, Conn (R.J., K.S.); and Baltimore VA Medical Center, Baltimore, Md (K.C.W.).
Radiology
|October 1, 2024
概括
有视觉的GPT-4 (GPT-4V) 在解释放射图像方面表现出与放射科医生相当的准确性. 人工智能辅助并没有改善人类的表现,GPT-4V在评估中严重依赖文本背景.
科学领域:
- 放射学中的人工智能
- 医学图像分析 医学图像分析
- 诊断的准确性 诊断的准确性
背景情况:
- 在医学评估中,GPT-4V显示出前景,但其放射图像解释能力尚未得到充分探索.
- 评估像GPT-4V这样的先进AI模型对于理解它们在临床环境中的作用至关重要.
研究的目的:
- 评估GPT-4V在使用组合文本和图像输入来解释放射病例的准确性.
- 为了比较GPT-4V与放射科医生和住院医生的性能.
- 为了确定GPT-4V辅助是否提高了人类诊断的准确性,并评估仅文本与仅图像的性能.
主要方法:
- 在观察员研究中使用了RSNA 2023的72个"当天案例"问题.
- GPT-4V评估了只有图像,只有文本和组合输入的案例.
- 放射科医生和住院人员回答了有或没有人工智能协助的病例.
主要成果:
- 通过组合输入,GPT-4V实现了43%的准确性.
- 放射科医生和住院人员的表现并没有明显优于GPT-4V.
- 只有文本 (50%) 的GPT-4V的准确性高于只有图像 (38%) 的输入,这表明它依赖于文本上下文.
- 人工智能辅助并没有提高人类阅读器的准确性.
结论:
- 在放射病例评估中,GPT-4V的准确性与人类专家相美.
- GPT-4V的性能受到文本上下文的显著影响.
- 目前来自GPT-4V的AI辅助并不能提高放射科医生或住院诊断的性能.
更多相关视频
相关概念视频
02:07Improving Translational Accuracy
Base complementarity between the three base pairs of mRNA codon and the tRNA anticodon is not a failsafe mechanism. Inaccuracies can range from a single mismatch to no correct base pairing at all. The free energy difference between the correct and nearly correct base pairs can be as small as 3 kcal/ mol. With complementarity being the only proofreading step, the estimated error frequency would be one wrong amino acid in every 100 amino acids incorporated. However, error frequencies observed in...
02:07Improving Translational Accuracy
Base complementarity between the three base pairs of mRNA codon and the tRNA anticodon is not a failsafe mechanism. Inaccuracies can range from a single mismatch to no correct base pairing at all. The free energy difference between the correct and nearly correct base pairs can be as small as 3 kcal/ mol. With complementarity being the only proofreading step, the estimated error frequency would be one wrong amino acid in every 100 amino acids incorporated. However, error frequencies observed in...