最佳的大型语言模型用于选系统审查的引用
Takehiko Oami1, Yohei Okada2,3, Taka-Aki Nakada1
1Department of Emergency and Critical Care Medicine, Chiba University Graduate School of Medicine, Chiba, Japan.
Research synthesis methods
|February 2, 2026
概括
这项研究比较了四个大型语言模型 (LLM) 用于系统性审查引文选. 双子1.5 Pro和Claude 3.5 Sonnet显示了更高的灵敏度,而GPT-4o和Llama 3.3 70B提供了更好的特异性,优化了审查工作流程.
科学领域:
- 人工智能的人工智能
- 医疗信息学 医疗信息学
- 医疗保健服务研究 医疗服务研究
背景情况:
- 大型语言模型 (LLM) 对自动化系统审查引文选充满希望.
- 不同的LLM对此任务的比较效率尚未得到充分证实.
- 系统性审查对于以证据为基础的医学至关重要,但耗时.
研究的目的:
- 在引用选中比较四个LLM (GPT-4o,Gemini 1.5 Pro,Claude 3.5 Sonnet,Llama 3.3 70B) 的准确性,效率,成本和一致性.
- 评估综合方法使用多个LLM对选绩效的影响.
- 评估LLM在简化系统审查工作流程方面的潜力.
主要方法:
- 四个LLM被用来选来自日本临床实践指南毒症和败血症休克2024的临床问题的引用.
- 对每个LLM计算了灵敏度,特异性,选时间,成本和一致性.
- 一个后期分析检查了结合LLM输出的效果.
主要成果:
- GPT-4o和Llama 3.3 70B的特异性很高,但敏感性较低.
- 双子 1.5 Pro 和克劳德 3.5 索内特在较低的特异性下实现了更高的灵敏度.
- GPT-4o是最快的,Llama 3.3 70B是最具成本效益的,并且跨模型的一致性是相似的.
- 整体方法提高了灵敏度,但增加了假阳性.
结论:
- 个别的LLM在引文选方面提供了独特的优势,节省时间和减少工作量.
- 虽然LLMs简化了查,但管理错误阳性仍然是一个挑战.
- 结合LLM可以提高灵敏度,突出其优化系统审查流程的潜力.
更多相关视频
相关概念视频
Language
913
Language is a unique communication system that uses words and systematic rules to organize and transmit information. Unlike other forms of communication, which may involve postures, movements, odors, or vocalizations, language relies on symbols and grammar. This makes human communication distinct from that of other species, who also communicate but do not use language in the same way humans do.
Corballis and Suddendorf (2007) and Tomasello and Rakoczy (2003) highlight the role of language in...
Corballis and Suddendorf (2007) and Tomasello and Rakoczy (2003) highlight the role of language in...
913
Review and Preview
8.4K
In statistics, several tools are used to interpret the data. Measures of central tendency represent the characteristics of the data, such as mean, median, and mode. Additionally, measures of variance like standard deviation and range are used to find the spread of data from the mean. Relative standing measures the distance between data locations. Commonly used measures of relative standings are percentile, z score, and quartiles.
Percentiles are a type of fractile that partition data into...
Percentiles are a type of fractile that partition data into...
8.4K
Review and Preview
11.3K
Data are individual items of information obtained from a population or sample. Data may be classified as qualitative (categorical), quantitative continuous, or quantitative discrete. Because it is not practical to measure the entire population in a study, researchers use samples to represent the population. A random sample is a representative group from the population chosen by using a method that gives each individual in the population an equal chance of being included in the sample. Random...
11.3K
Components of Language
821
Language, whether spoken, signed, or written, consists of specific components: lexicon and grammar. The lexicon is the vocabulary of a language, comprising its words. Grammar is the set of rules used to convey meaning through the lexicon. For example, English grammar adds “-ed” to most verbs to indicate past tense. Words are formed by combining phonemes, which are the basic sound units of a language. Different languages have different sets of phonemes (e.g., “ah” vs.
821
Language Development
912
Children master language quickly and with relative ease, supported by both biological predisposition and reinforcement. B. F. Skinner (1957) proposed that language is learned through reinforcement, while Noam Chomsky (1965) argued that language acquisition mechanisms are biologically determined.
The critical period for language acquisition suggests that the ability to acquire language is at its peak early in life. As people age, this proficiency decreases. Language development begins very...
The critical period for language acquisition suggests that the ability to acquire language is at its peak early in life. As people age, this proficiency decreases. Language development begins very...
912
Language and Cognition
800
Language serves as a bridge between ideas and communication, influencing how individuals perceive and interact with the world. Psychologists have long debated whether language shapes thought or vice versa. This discussion gained grip with Edward Sapir and Benjamin Lee Whorf in the 1940s, who proposed that language determines thought, a concept known as linguistic determinism. They suggested that the vocabulary and structure of a language influence how its speakers think and perceive reality.
800


