相关实验视频

Updated: Jun 22, 2025

Augmenting Large Language Models via Vector Embeddings to Improve Domain-Specific Responsiveness

Augmenting Large Language Models via Vector Embeddings to Improve Domain-Specific Responsiveness

Published on: December 6, 2024

评估ChatGPT在科学研究中模仿人类审稿人的能力:一种描述性和定性方法

Aiman Suleiman¹, Dario von Wedel², Ricardo Munoz-Acuna²

¹Department of Anesthesia, Critical Care and Pain Medicine, Harvard Medical School, Beth Israel Deaconess Medical Center, Boston, MA, USA; Center for Anesthesia Research Excellence (CARE), Harvard Medical School, Beth Israel Deaconess Medical Center, Boston, MA, USA; Department of Anesthesia, Critical Care and Pain Medicine, Albert Einstein College of Medicine, Montefiore Medical Center, Bronx, NY, USA.

Computer methods and programs in biomedicine

|July 2, 2024

概括

聊天GPT (生成预训练变压器) 在科学同行评审中表现出有限的能力,以模仿人类评审者. 这种人工智能工具与大多数人类的评论不一致,这表明它还不是在评估研究中可靠的替代人类专业知识.

科学领域:

医学研究医学研究
科学出版业的科学出版.
科学领域的人工智能

背景情况:

人工智能,特别是ChatGPT在科学同行评审中的作用正在扩大.
人工智能产生的评论中存在潜在的偏见和不准确性的担忧.
这项研究评估了ChatGPT复制人类审查员评估的能力.

研究的目的:

质量评估聊天GPT (GPT-4) 和人类审查员的评论之间的协议.
确定人工智能在科学手稿评价中能在多大程度上效仿人类判断力.

主要方法:

分析了来自知名医学期刊的20篇原创研究文章.
对比ChatGPT-4的定性评论与每篇文章至少有三位人类评论员的评论.
协议级别的分类:完全,部分,不存在或矛盾.

主要成果:

发现了显著的不同意度:78.5%的ChatGPT评论显示没有与人类审稿人达成一致.
只有在6.7%的案例中观察到完全一致.
人工智能评论对方法的同意率最低 (3.6%),对一般手稿评论的同意率最高 (22.1%).

结论:

聊天GPT (GPT-4) 在科学研究同行评审中表现出有限的能力,以模仿人类审稿人.
当前的人工智能模型还不足以取代科学手稿的批判性评估中的人类专业知识.

关键词:

人工智能的人工智能这是一个偏见的偏见.聊天GPT 聊天GPT 聊天同行评审是一项同行评审.质量协议是质量协议.

更多相关视频

Assessment of Mouse Judgment Bias through an Olfactory Digging Task

Assessment of Mouse Judgment Bias through an Olfactory Digging Task

Published on: March 4, 2022

Measuring Neural and Behavioral Activity During Ongoing Computerized Social Interactions: An Examination of Event-Related Brain Potentials

Measuring Neural and Behavioral Activity During Ongoing Computerized Social Interactions: An Examination of Event-Related Brain Potentials

Published on: November 15, 2014

相关概念视频

Stereotype Content Model

Stereotype Content Model

The Stereotype Content Model (SCM) was first proposed by Susan Fiske and her colleagues (Fiske, Cuddy, Glick & Xu, 2002; see also Fiske, 2012 and Fiske, 2017). The SCM specifies that when someone encounters a new group, they will stereotype them based on two metrics: warmth—or that group’s perceived intent, and how likely they are to provide help or inflict harm—and competence—or their ability to carry out that objective. Depending on the warmth-competence...