人为主导和人工智能自动化系统性审查的批判性评估:比较评估
Lucija Gosak1, Gregor Štiglic2, Wilson Wai San Tam3
1Faculty of Health Sciences, University of Maribor, Maribor 2000, Slovenia.
Nurse education in practice
|November 7, 2025
概括
人工智能 (AI) 大型语言模型可以协助证据的批判性评估,但不能取代人类的判断. 人类监督对于基于证据的复杂实践决策至关重要.
科学领域:
- 医疗信息学 医疗信息学
- 医疗保健中的人工智能
- 基于证据的实践.
背景情况:
- 对证据的批判性评估对医疗保健专业人员至关重要,但许多人缺乏必要的技能.
- 大型语言模型 (LLM) 为支持批判性评估提供了潜在的解决方案,但它们的有效性尚未得到充分证实.
- 本研究探讨了LLMs在协助科学文献的批判性评价方面的有用性.
研究的目的:
- 为了比较人类审稿人的表现与各种AI大型语言模型在批判性评估系统性评论.
- 为了确定人工智能和人类对证据质量的评估之间的一致程度.
- 确定AI在关键评估任务中脱而出或扎的特定领域.
主要方法:
- 一项比较研究旨在评估五个LLM (克劳德3.7索内特,双子星2.0闪光,GPT-4.5,Grok-2,困惑声纳Pro) 与两个人类审查员.
- 使用JBI关键评估检查清单评估了四项专注于药物错误减少干预措施的系统审查.
- 使用Cohen's Kappa量化了协议,并对差异进行了描述性分析.
主要成果:
- 在JBI的11个检查清单项目中,LLMs在5个问题上与人类审核员达成完全一致.
- 在评估搜索策略,纳入标准和出版偏见方面,分歧最为频繁.
- 根据LLM的协议水平有所不同,Claude 3.7 Sonnet显示了最高的协议 (κ = 0.732) 和Gemini 2.0 Flash显示了最低的协议 (κ = 0.394).
结论:
- 法律法学可以为批判性评估的某些方面提供支持,但缺乏对复杂的方法论判断的细微理解.
- 虽然一些人工智能模型与人类专家有适度的一致性,但它们不能取代人类的批判性思维.
- 人类监督是不可或缺的,将LLM定位为有价值的辅助,而不是取代基于证据的实践.
更多相关视频
05:47Evidence-based Knowledge Synthesis and Hypothesis Validation: Navigating Biomedical Knowledge Bases via Explainable AI and Agentic Systems
Published on: June 13, 2025
1.3K
07:08Estimate the Cognitive Load Using Electrocardiographic Measure: A Human-AI Collaborative Task
Published on: December 5, 2025
166
相关概念视频
01:24Humanistic Psychology
2.4K
Humanistic psychology emerged in the mid-20th century as a response to the deterministic and pessimistic nature of behaviorism and psychoanalysis. While behaviorism focused on observable behaviors influenced by the environment and psychoanalysis delved into unconscious motivations, both theories suggested that human actions lacked free will. In contrast, humanistic psychology offers a perspective that emphasizes the innate potential for goodness and growth within every individual.
This approach...
This approach...
2.4K
01:20Introduction to Cognitive Psychology
2.2K
Cognitive psychology is the field of psychology dedicated to examining how people think. It attempts to explain how and why we think the way we do by studying the interactions among human thinking, emotion, creativity, language, and problem-solving, as well as other cognitive processes. Cognitive psychology studies how information is processed and manipulated in remembering, thinking, and knowing.
This field emerged in the mid-20th century, following a period dominated by behaviorism, which...
This field emerged in the mid-20th century, following a period dominated by behaviorism, which...
2.2K
01:16Non-equilibrium in the Cell
5.3K
An important concept in studying metabolism and energy is that of chemical equilibrium. Most chemical reactions are reversible. They can proceed in both directions, releasing energy into their environment in one direction, and absorbing it from the environment in the other direction. The same is true for the chemical reactions involved in cell metabolism, such as the breaking down and building up of proteins into and from individual amino acids, respectively. Reactants within a closed system...
5.3K
01:37Reason and Intuition
7.4K
The human brain processes information for decision-making using one of two routes: an intuitive system and a rational system (Epstein, 1994; popularized by Kahneman, 2011 as System 1 and System 2, respectively). The intuitive system is quick, impulsive, and operates with minimal effort, relying on emotions or habits to provide cues for what to do next, while the rational system is logical, analytical, deliberate, and methodical. Research in neuropsychology suggests that the...
7.4K