使用大型语言模型来解决系统性审查中的上下文问题
Susanne Hempel1, Kimny Sysawang1, Haley K Holmer2
1Southern California Evidence Review Center University of Southern California Los Angeles California USA.
Cochrane evidence synthesis and methods
|March 2, 2026
概括
大型语言模型 (LLM) 可以在系统性审查中切实回答上下文问题,提供明确和临床可信的答案. 然而,人类的专业知识对于验证信息和确保有意义地使用LLM产生的内容至关重要.
科学领域:
- 医疗服务研究 医疗服务研究
- 医疗保健中的人工智能
- 基于证据的实践实践.
背景情况:
- 医疗保健研究和质量机构 (AHRQ) 基于证据的实践中心 (EPC) 计划的系统证据审查 (SER) 使用上下文问题作为背景.
- 目前还缺乏一种标准化的方法来解决系统性审查中的这些上下文问题.
研究的目的:
- 探索使用公开可用的大型语言模型 (LLM) 来解决系统审查中的上下文问题的可行性和有效性.
- 为了将LLM生成的响应与SER中人类生成的响应进行比较.
主要方法:
- 从25个SER中选择的上下文问题 (20个已发表,5个未发表).
- 使用的LLM (ChatGPT,Bard,Claude,Perplexity) 使用最少的提示工程来生成答案.
- 两个独立审稿人根据预先定义的标准评估了LLM的回应,评估了可行性,内容/结构有效性,错误,偏见和一致性.
主要成果:
- 在回答上下文问题时,LLM 证明了其可行性,其答案是明确的,临床上可信的和结构良好的.
- 答案在内容和格式上有所不同,并且由于定期更新LLM,无法重现.
- 检测到的实际错误或偏见很少,但引用往往无法验证 ("伪造").
- 法律法学士的答案提供了更多的背景,而SER提供了更细致的答案;一致性各不相同.
- 增量有效性结果是混合的,可能依赖于工具.
结论:
- 大型语言模型显示了在系统性审查中帮助解决上下文问题的潜力.
- 人类专业知识对于批判性评估和有意义地将LLM产生的信息整合到系统审查中至关重要.
相关概念视频
Language and Cognition
891
Language serves as a bridge between ideas and communication, influencing how individuals perceive and interact with the world. Psychologists have long debated whether language shapes thought or vice versa. This discussion gained grip with Edward Sapir and Benjamin Lee Whorf in the 1940s, who proposed that language determines thought, a concept known as linguistic determinism. They suggested that the vocabulary and structure of a language influence how its speakers think and perceive reality.
891
Language Development
1.0K
Children master language quickly and with relative ease, supported by both biological predisposition and reinforcement. B. F. Skinner (1957) proposed that language is learned through reinforcement, while Noam Chomsky (1965) argued that language acquisition mechanisms are biologically determined.
The critical period for language acquisition suggests that the ability to acquire language is at its peak early in life. As people age, this proficiency decreases. Language development begins very...
The critical period for language acquisition suggests that the ability to acquire language is at its peak early in life. As people age, this proficiency decreases. Language development begins very...
1.0K
Strategies for Assessing and Addressing Confounding
491
Confounding is a critical issue in epidemiological studies, often leading to misleading conclusions about associations between exposures and outcomes. It occurs when the relationship between the exposure and the outcome is mixed with the effects of other factors that influence the outcome. Given that, addressing confounding is of high importance for drawing accurate inferences in research.
Confounding can be addressed at both the design phase of a study and through analytical methods after data...
Confounding can be addressed at both the design phase of a study and through analytical methods after data...
491
Observational Studies
11.2K
Observational studies are a type of analytical study where researchers observe events without any interventions. In other words, the researcher does not influence the response variable or the experiment's outcome.
There are three types of observational studies – Prospective, retrospective, and cross-sectional.
Prospective Study
Prospective studies, also known as longitudinal or cohort studies, are carried out by collecting future data from groups sharing similar characteristics. One...
There are three types of observational studies – Prospective, retrospective, and cross-sectional.
Prospective Study
Prospective studies, also known as longitudinal or cohort studies, are carried out by collecting future data from groups sharing similar characteristics. One...
11.2K
Systematic Error: Methodological and Sampling Errors
11.2K
In the case of systematic errors, the sources can be identified, and the errors can be subsequently minimized by addressing these sources. According to the source, systematic errors can be divided into sampling, instrumental, methodological, and personal errors.
Sampling errors originate from improper sampling methods or the wrong sample population. These errors can be minimized by refining the sampling strategy. Defective instruments or faulty calibrations are the sources of instrumental...
Sampling errors originate from improper sampling methods or the wrong sample population. These errors can be minimized by refining the sampling strategy. Defective instruments or faulty calibrations are the sources of instrumental...
11.2K
Typical Model Studies
662
Fluid mechanics model studies often utilize scaled-down systems to predict fluid behavior in full-scale environments, such as river flows, dam spillways, and structures interacting with open surfaces. Maintaining Froude number similarity in river models is crucial, as it replicates surface flow features like wave patterns and velocities.
662

