从算法到学术:与人类标准对比人工智能生成的科学论文的努力
Jackson Woodrow1, Nour Nassour1, John Y Kwon1
1Foot & Ankle Research and Innovation Lab (FARIL), Department of Orthopaedic Surgery, Massachusetts General Hospital, Harvard Medical School, Boston, MA, USA.
The archives of bone and joint surgery
|May 7, 2025
概括
人类撰写的科学论文在可读性和接受性方面通常优于人工智能生成的内容. 虽然人工智能看起来很有前途,但严格的事实核查对于科学完整性至关重要.
科学领域:
- 医学研究 医学研究
- 科学写作 科学写作
- 人工智能的人工智能是人工智能.
背景情况:
- 人工智能语言模型越来越多地用于科学写作.
- 评估人工智能产生的科学内容的质量和准确性至关重要.
研究的目的:
- 量化比较人工智能生成的科学论文与人类撰写的科学论文的准确性,可读性和接受可能性.
- 评估人工智能生成内容的事实准确性和引用精度.
主要方法:
- 创建了六篇论文:两个是ChatGPT,两个是Assistant by scite,两个是人类在类似主题上的论文.
- 六位独立的审稿人,盲目对作者的认知,对论文的小部分 (1-4) 进行了评分,并猜测了作者.
- 研究作者还评估了人工智能生成的论文的事实准确性和引用正确性.
主要成果:
- 人类撰写的论文获得了更高的整体分数,骨骨折论文具有统计学意义 (P=0.028对比ChatGPT,P=0.043对比助理).
- 人工智能论文的事实准确度在94.98%至100%之间,ChatGPT的骨关节炎评论达到100%.
- 引用准确度有很大的差异,ChatGPT的关节骨关节炎论文为90%,Assistant的骨骨折论文为39.68%.
结论:
- 人工智能显示出科学知识共享的潜力,但需要负责任地使用.
- 在使用人工智能生成的内容时,全面的事实核查对于维护科学话语的完整性至关重要.
相关概念视频
01:16Non-equilibrium in the Cell
4.1K
An important concept in studying metabolism and energy is that of chemical equilibrium. Most chemical reactions are reversible. They can proceed in both directions, releasing energy into their environment in one direction, and absorbing it from the environment in the other direction. The same is true for the chemical reactions involved in cell metabolism, such as the breaking down and building up of proteins into and from individual amino acids, respectively. Reactants within a closed system...
4.1K
01:52Regression Toward the Mean
6.2K
Regression toward the mean (“RTM”) is a phenomenon in which extremely high or low values—for example, and individual’s blood pressure at a particular moment—appear closer to a group’s average upon remeasuring. Although this statistical peculiarity is the result of random error and chance, it has been problematic across various medical, scientific, financial and psychological applications. In particular, RTM, if not taken into account, can interfere when...
6.2K
01:29Measures of Intelligence
5.2K
Psychologists measure intelligence by using standardized tests that produce a score known as the intelligence quotient or IQ. To understand IQ tests, it's important to recognize the key principles behind their construction: validity, reliability, and standardization.
Validity refers to how well a test measures what it claims to measure. An intelligence test should accurately assess intelligence rather than another characteristic, like anxiety. Criterion validity is one way to evaluate this;...
Validity refers to how well a test measures what it claims to measure. An intelligence test should accurately assess intelligence rather than another characteristic, like anxiety. Criterion validity is one way to evaluate this;...
5.2K