探索ChatGPT的AI幻觉:ChatGPT模型和培训条件的参考准确性和引用相关性
Adam Cheng1, Vikhashni Nagesh, Susan Eller
1From the KidSIM Simulation Program (A.C.), Alberta Children's Hospital, Departments of Pediatrics and Emergency Medicine, Cumming School of Medicine, University of Calgary, Calgary, Alberta, Canada; Department of Pediatrics (V.N.), Cumming School of Medicine, University of Calgary, Calgary, Alberta, Canada; Center for Immersive and Simulation-Based Learning (S.E.), Stanford School of Medicine, Stanford, CA; Departments of Pediatrics and Emergency Medicine (V.G.), Cumming School of Medicine, University of Calgary, Calgary, Alberta, Canada; and KidSIM Simulation Program (Y.L.), Alberta Children's Hospital, Calgary, Alberta, Canada.
像ChatGPT这样的生成人工智能工具在医疗保健模拟文章中引用引用时不可靠. 培训提示不会提高这些AI模型生成的引用的准确性或相关性.
科学领域:
- 医疗教育中的人工智能
- 医疗保健模拟研究研究 医疗保健模拟研究
- 学术出版学术出版
背景情况:
- 生成性AI工具,如ChatGPT,越来越多地用于协助学术写作.
- 在医疗保健模拟文献中,人工智能生成的引用的可靠性在很大程度上是未知的.
- 本研究调查了AI在学术手稿中提供的参考文献的准确性和相关性.
研究的目的:
- 评估ChatGPT生成的关于医疗保健模拟汇报的文章的参考准确性和引用相关性.
- 确定快速工程是否可以提高AI生成的学术内容的引用质量.
- 为了比较不同ChatGPT模型 (ChatGPT-4和ChatGPT o1) 在学术写作辅助方面的性能.
主要方法:
- 聊天GPT-4和聊天GPT o1模型产生了关于医疗保健模拟汇报的学术文章.
- 文章是在每个模型的各种快速培训条件下创建的.
- 两个盲目审稿人独立评估了所有生成的引用和引用的准确性和相关性.
主要成果:
- 在ChatGPT模型和培训条件中共生成了15篇文章.
- 在303个生成的引用中,只有60.4%是准确的,模型或提示之间没有显著差异.
- 在451个引文中,仅有22.2%被认为是高度相关的,并且在培训方面也没有显著的改善.
结论:
- 无论是ChatGPT-4还是ChatGPT o1,都表明在生成医疗保健模拟解答文章的准确和相关引用方面不可靠.
- 目前的快速培训方法并不能显著提高AI生成的学术内容的引用准确性或相关性.
- 研究人员在使用人工智能工具进行学术写作时应谨慎使用,特别是在参考完整性方面.
相关概念视频
Improving Translational Accuracy
Accuracy and Errors in Hypothesis Testing
In hypothesis testing, the probability of making a Type I error, denoted as α, is commonly set at 0.05. This significance level indicates a 5%...
The Representativeness Heuristic
Prediction Intervals
However, the point estimate is most likely not the exact value of the population parameter, but close to it. After calculating point estimates, we construct interval estimates, called confidence intervals or prediction intervals. This prediction interval comprises a range of values unlike the point estimate and is a better predictor of the observed sample value, y.
Hindsight Biases
Neural Circuits
Neuronal pools are collections of nerve cells with similar functions and interact through chemical and electrical signals. These pools include both interneurons (the central neural circuit nodes that...


