人与人工智能产生的关节整形术文献:对感知到的沟通,质量和作者身份进行单盲分析
Kyle W Lawrence1, Akram A Habibi1, Spencer A Ward1
1Department of Orthopedic Surgery, NYU Langone Health, New York, New York, USA.
概括
大型语言模型 (LLM) 生成的摘要很难与人写的摘要区分,但被认为质量较差. 在科学写作中使用人工智能时建议谨慎.
科学领域:
- 医疗研究中的人工智能
- 在科学出版中使用自然语言处理.
背景情况:
- 大型语言模型 (LLM) 对医学研究的影响在很大程度上仍未被探索.
- 与人类撰写的摘要相比,评估LLM生成的摘要的区分和质量至关重要.
研究的目的:
- 为了确定骨科外科医生是否能区分人工智能生成和人类撰写的摘要.
- 为了比较由大型语言模型 (LLM) 生成的摘要与人类撰写的摘要的感知质量.
主要方法:
- 使用ChatGPT (人工智能生成) 创建了20个关节整形摘要,并与原始人写的摘要进行了比较.
- 六位失明的骨科外科医生评估了摘要的整体质量,沟通清晰度和对作者信心.
主要成果:
- 人类撰写的摘要获得了较高的作者信任分数 (p=0.028).
- 人工智能生成的摘要的作者信任分数在统计学上无法区分于随机机会 (p=0.999).
- 人类撰写的摘要在整体质量方面被评为显著更高 (p=0.019).
结论:
- 虽然人工智能生成的摘要显示出作者鉴别的困难,但它们与人类撰写的摘要的质量不匹配.
- 这些发现表明,在将LLMs整合到科学写作过程中时,需要谨慎.
- 需要进一步的研究,以了解人工智能在医学研究和出版方面的全部影响.
相关概念视频
Reason and Intuition
The human brain processes information for decision-making using one of two routes: an intuitive system and a rational system (Epstein, 1994; popularized by Kahneman, 2011 as System 1 and System 2, respectively). The intuitive system is quick, impulsive, and operates with minimal effort, relying on emotions or habits to provide cues for what to do next, while the rational system is logical, analytical, deliberate, and methodical. Research in neuropsychology suggests that the brain can only use...
Stereotype Content Model
The Stereotype Content Model (SCM) was first proposed by Susan Fiske and her colleagues (Fiske, Cuddy, Glick & Xu, 2002; see also Fiske, 2012 and Fiske, 2017). The SCM specifies that when someone encounters a new group, they will stereotype them based on two metrics: warmth—or that group’s perceived intent, and how likely they are to provide help or inflict harm—and competence—or their ability to carry out that objective. Depending on the warmth-competence categorization, a person will feel...
Non-equilibrium in the Cell
An important concept in studying metabolism and energy is that of chemical equilibrium. Most chemical reactions are reversible. They can proceed in both directions, releasing energy into their environment in one direction, and absorbing it from the environment in the other direction. The same is true for the chemical reactions involved in cell metabolism, such as the breaking down and building up of proteins into and from individual amino acids, respectively. Reactants within a closed system...
Social Foundations of Self IV: Self in Digital Communication
Since the early 2000s, computer-mediated communication (CMC) has grown rapidly, playing a crucial role in self-development. A key distinction between CMC and real-life interactions is the lack of a physically present partner. This absence makes non-verbal cues such as facial expressions, body language, and paralinguistic signals unavailable in CMC platforms like email, instant messaging, or social media. The lack of these cues can create ambiguity and complicate how feedback is interpreted.The...
Automatic Processing and Automatic Social Behavior
Automatic processing refers to the cognitive operations that occur without conscious intent or awareness, playing a fundamental role in shaping social cognition and behavior. These processes enable individuals to navigate complex social environments efficiently by relying on mental shortcuts and pre-existing knowledge structures known as schemas. One of the most influential mechanisms underlying automatic processing is priming, which subtly activates mental representations through exposure to...
Understanding Deception
Deception is a pervasive aspect of human communication. Empirical studies have shown that most individuals engage in some form of deceit on a daily basis, with approximately 20% of social exchanges involving deceptive elements. Lying follows a developmental trajectory, peaking during adolescence and declining with age, possibly due to the maturation of cognitive control and social accountability.Cognitive and Social Factors in Deception DetectionDespite its prevalence, accurately detecting...


