CARDBiomedBench:用于评估生物医学研究中大型语言模型的性能的一个基准
Owen Bianchi1, Maya Willey1, Chelsea X Alvarado1
1Center for Alzheimer's and Related Dementias, National Institute on Aging, National Institutes of Health, Bethesda, MD, USA; DataTecnica, Washington, DC, USA.
The Lancet. Digital health
|February 1, 2026
概括
大型语言模型 (LLM) 在生物医学研究中显示出潜力,但在准确性方面存在困难. 一个新的基准,CARDBiomedBench,揭示了目前LLM对神经退行性疾病研究的推理中的重大差距.
科学领域:
- 生物医学科学 生物医学科学
- 人工智能的人工智能
- 神经退行性疾病研究研究
背景情况:
- 大型语言模型 (LLM) 为生物医学研究提供了变革性的潜力.
- 然而,它们在复杂,数据密集的科学领域进行准确推理的能力尚未确定.
- 评估LLM在神经退行性疾病等专业领域的表现至关重要.
研究的目的:
- 推出CARDBiomedBench,这是一个大规模的问答基准,用于评估生物医学科学LLM.
- 评估最先进的LLM的推理能力,特别是在神经退行性疾病研究中.
- 确定目前在科学应用中LLM准确性和安全性的局限性.
主要方法:
- CARDBiomedBench包括超过68000个由专家注释和数据增强获得的精心策划的问答对.
- 问题涵盖十个生物类别和九种推理类型,利用诸如GWAS和药物数据库之类的资源.
- 使用BioScore评估模型响应,测量准确度 (响应质量率,RQR) 和安全性 (避免错误答案).
主要成果:
- 测试了18个LLM,发现了显著的绩效差距.
- 克劳德-3.5-索内特显示了高安全性但低准确性 (75%的安全性,24%的RQR).
- GPT-4.1的准确性更高,但安全性较差 (7%的安全性,51%的RQR);没有一个模型有效地平衡了这两种指标.
结论:
- CARDBiomedBench建立了一个评估生物医学LLM的新标准.
- 当前的LLM在科学推理中平衡准确性和安全性方面存在局限性.
- 这一基准有助于开发更可靠的科学研究人工智能系统.
相关概念视频
Language
913
Language is a unique communication system that uses words and systematic rules to organize and transmit information. Unlike other forms of communication, which may involve postures, movements, odors, or vocalizations, language relies on symbols and grammar. This makes human communication distinct from that of other species, who also communicate but do not use language in the same way humans do.
Corballis and Suddendorf (2007) and Tomasello and Rakoczy (2003) highlight the role of language in...
Corballis and Suddendorf (2007) and Tomasello and Rakoczy (2003) highlight the role of language in...
913
Components of Language
821
Language, whether spoken, signed, or written, consists of specific components: lexicon and grammar. The lexicon is the vocabulary of a language, comprising its words. Grammar is the set of rules used to convey meaning through the lexicon. For example, English grammar adds “-ed” to most verbs to indicate past tense. Words are formed by combining phonemes, which are the basic sound units of a language. Different languages have different sets of phonemes (e.g., “ah” vs.
821
Language Development
912
Children master language quickly and with relative ease, supported by both biological predisposition and reinforcement. B. F. Skinner (1957) proposed that language is learned through reinforcement, while Noam Chomsky (1965) argued that language acquisition mechanisms are biologically determined.
The critical period for language acquisition suggests that the ability to acquire language is at its peak early in life. As people age, this proficiency decreases. Language development begins very...
The critical period for language acquisition suggests that the ability to acquire language is at its peak early in life. As people age, this proficiency decreases. Language development begins very...
912
Language and Cognition
800
Language serves as a bridge between ideas and communication, influencing how individuals perceive and interact with the world. Psychologists have long debated whether language shapes thought or vice versa. This discussion gained grip with Edward Sapir and Benjamin Lee Whorf in the 1940s, who proposed that language determines thought, a concept known as linguistic determinism. They suggested that the vocabulary and structure of a language influence how its speakers think and perceive reality.
800
Self-Evaluation Maintenance Model
311
The Self-Evaluation Maintenance (SEM) model offers a psychological framework to understand how individuals’ self-esteem is influenced by the achievements of others, particularly those with whom they share close personal bonds. The SEM model operates when personal rather than social identity guides individuals. Central to this model is the notion that individuals have an inherent desire to preserve a favorable self-image, which is continuously shaped by interpersonal comparisons and...
311
Higher Mental Functions of the Brain: Language
3.8K
Language is a system of communication that allows the expression of thoughts, ideas, and feelings. The brain processes language in both hemispheres.
Language formation and comprehension take place in the dominant hemisphere. The dominant hemisphere is responsible for understanding the meaning of spoken, written, or sign language, as well as the ability to communicate. For most people, the left hemisphere is the dominant one. The right hemisphere, then, gives tone and emotional context to the...
Language formation and comprehension take place in the dominant hemisphere. The dominant hemisphere is responsible for understanding the meaning of spoken, written, or sign language, as well as the ability to communicate. For most people, the left hemisphere is the dominant one. The right hemisphere, then, gives tone and emotional context to the...
3.8K


