在范围审查中利用开源大型语言模型 (LLM):关于残疾和AI应用的案例研究.
Azadeh Bayani1, Leandre Parfait Epoh Ewane2, Davllyn Santos Oliveira Dos Anjos1
1Centre de recherche en santé publique, Université de Montréal et CIUSSS du Centre-Sud-de-l'Île-de-Montréal, Montréal, Canada; Laboratoire Transformation Numérique en Santé (LabTNS), Québec, Canada.
International journal of medical informatics
|July 29, 2025
概括
大型语言模型 (LLM) 显示出自动化科学审查的前景,但人类监督对于最佳结果至关重要. 将LLM与人类判断相结合,提高了审查准确性和可重复性.
科学领域:
- 科学方法科学方法学
- 在研究中的人工智能.
背景情况:
- 大型语言模型 (LLM) 提供了在科学评论中自动化手工任务的潜力.
- 任务包括数据提取,文献选,总结和质量评估.
研究的目的:
- 评估在标题/摘要选和全文数据提取中LLM的表现,以进行范围审查.
- 评估LLM的有效性,效率和与人类任务的整合.
主要方法:
- 自动标题/摘要选,全文选和数据提取 (9维).
- 评估了四个开源的LLM:Mistral,Vicuna和Llama 3.2 (1B,3B参数).
主要成果:
- 拉玛3.2-3B在标题/摘要选中实现了66%的准确性,在全文选中达到65%.
- 米斯特拉尔在大多数维度的数据提取方面表现出色;拉玛3.2-3B在目标和影响方面表现最好.
结论:
- 在自动化范围审查方面,LLM具有潜力,但存在局限性.
- 完全自动化是次优的;混合方法平衡了LLM的优势与人类的判断力.
- 控制整合支持复制,改进和跟进审查.
相关概念视频
Learning Disabilities
272
Learning disabilities are cognitive disorders caused by neurological impairments that affect cognitive functions like language and reading, without indicating overall intellectual or developmental challenges. These disabilities differ from global intellectual or developmental disabilities as they are limited to distinct cognitive functions. Common learning disabilities include dysgraphia, dyslexia, and dyscalculia, each of which impacts unique aspects of learning.
Dyslexia
Dyslexia is a...
Dyslexia
Dyslexia is a...
272
Stereotype Content Model
14.9K
The Stereotype Content Model (SCM) was first proposed by Susan Fiske and her colleagues (Fiske, Cuddy, Glick & Xu, 2002; see also Fiske, 2012 and Fiske, 2017). The SCM specifies that when someone encounters a new group, they will stereotype them based on two metrics: warmth—or that group’s perceived intent, and how likely they are to provide help or inflict harm—and competence—or their ability to carry out that objective. Depending on the warmth-competence...
14.9K
Improving Translational Accuracy
11.9K
Base complementarity between the three base pairs of mRNA codon and the tRNA anticodon is not a failsafe mechanism. Inaccuracies can range from a single mismatch to no correct base pairing at all. The free energy difference between the correct and nearly correct base pairs can be as small as 3 kcal/ mol. With complementarity being the only proofreading step, the estimated error frequency would be one wrong amino acid in every 100 amino acids incorporated. However, error frequencies observed in...
11.9K
Language and Cognition
444
Language serves as a bridge between ideas and communication, influencing how individuals perceive and interact with the world. Psychologists have long debated whether language shapes thought or vice versa. This discussion gained grip with Edward Sapir and Benjamin Lee Whorf in the 1940s, who proposed that language determines thought, a concept known as linguistic determinism. They suggested that the vocabulary and structure of a language influence how its speakers think and perceive reality.
444
Language Development
454
Children master language quickly and with relative ease, supported by both biological predisposition and reinforcement. B. F. Skinner (1957) proposed that language is learned through reinforcement, while Noam Chomsky (1965) argued that language acquisition mechanisms are biologically determined.
The critical period for language acquisition suggests that the ability to acquire language is at its peak early in life. As people age, this proficiency decreases. Language development begins very...
The critical period for language acquisition suggests that the ability to acquire language is at its peak early in life. As people age, this proficiency decreases. Language development begins very...
454
Models, Theories, and Laws
7.0K
Scientists frequently use models to help them comprehend a specific collection of phenomena. In physics, a model is a condensed version of a physical system that is too complex to study thoroughly. One such example is the light wave model; unlike water waves, light waves are typically invisible to us. Nonetheless, it is helpful to think of light as being composed of waves, since investigations show that light behaves like water waves. Since it is impossible to visually see what is genuinely...
7.0K


