相关实验视频
Updated: Jan 22, 2026

05:31
Examining Bilingual Language Control Using the Stroop Task
Published on: February 26, 2020
15.5K
用MedHELM进行医疗任务的大型语言模型的整体评估
Suhana Bedi1, Hejie Cui2, Miguel Fuentes2
1Stanford University School of Medicine, Stanford, CA, USA. suhana@stanford.edu.
Nature medicine
|January 20, 2026
概括
MedHELM是一个评估医疗AI的新框架. 它使用临床医生验证的分类学和基准套件来比较大型语言模型 (LLM) 与现实世界的临床任务.
科学领域:
- 人工智能在医学中的应用
- 医疗信息学 医疗信息学
- 临床AI评估临床AI评估
背景情况:
- 目前对医学中的大型语言模型 (LLM) 的评估,例如许可证考试,无法捕捉现实世界临床实践的复杂性.
- 需要一个全面和可扩展的框架来评估医疗人工智能系统在各种临床任务中的性能.
研究的目的:
- 介绍MedHELM,一个医学AI的评估框架.
- 开发医学AI应用的临床医生验证的分类学.
- 创建一个基准套件,并系统地比较临床任务上的前沿LLM.
主要方法:
- 开发了医学AI应用的临床医生验证的分类法,分为五个类别和22个子类别.
- 构建了一个基准套件,包括涵盖所有子类别的37项评估.
- 采用了一种自动化的LLM陪审团评估方法来比较九个边境LLM.
主要成果:
- 先进的推理模型 (DeepSeek R1,o3-mini) 显示出优异的性能,获胜率为66%.
- 克劳德 3.5 索内特以15%的较低计算成本实现了类似的结果.
- MedHELM框架促进了LLM在医疗应用中的能力的系统比较.
结论:
- MedHELM提供了一种强大的方法来评估医疗人工智能系统.
- 该框架允许基于证据的医疗保健人工智能工具的选择.
- 目前的前沿LLM表现不同,先进的推理模型优秀,但成本效益也是一个关键因素.
相关概念视频
Language
892
Language is a unique communication system that uses words and systematic rules to organize and transmit information. Unlike other forms of communication, which may involve postures, movements, odors, or vocalizations, language relies on symbols and grammar. This makes human communication distinct from that of other species, who also communicate but do not use language in the same way humans do.
Corballis and Suddendorf (2007) and Tomasello and Rakoczy (2003) highlight the role of language in...
Corballis and Suddendorf (2007) and Tomasello and Rakoczy (2003) highlight the role of language in...
892
Components of Language
769
Language, whether spoken, signed, or written, consists of specific components: lexicon and grammar. The lexicon is the vocabulary of a language, comprising its words. Grammar is the set of rules used to convey meaning through the lexicon. For example, English grammar adds “-ed” to most verbs to indicate past tense. Words are formed by combining phonemes, which are the basic sound units of a language. Different languages have different sets of phonemes (e.g., “ah” vs.
769
Language Development
867
Children master language quickly and with relative ease, supported by both biological predisposition and reinforcement. B. F. Skinner (1957) proposed that language is learned through reinforcement, while Noam Chomsky (1965) argued that language acquisition mechanisms are biologically determined.
The critical period for language acquisition suggests that the ability to acquire language is at its peak early in life. As people age, this proficiency decreases. Language development begins very...
The critical period for language acquisition suggests that the ability to acquire language is at its peak early in life. As people age, this proficiency decreases. Language development begins very...
867
Language and Cognition
722
Language serves as a bridge between ideas and communication, influencing how individuals perceive and interact with the world. Psychologists have long debated whether language shapes thought or vice versa. This discussion gained grip with Edward Sapir and Benjamin Lee Whorf in the 1940s, who proposed that language determines thought, a concept known as linguistic determinism. They suggested that the vocabulary and structure of a language influence how its speakers think and perceive reality.
722
Inhaled Medications
761
Inhaled medications are crucial for managing chronic obstructive pulmonary disease (COPD) and asthma. They are essential for effective treatment and control, ensuring optimal respiratory health and well-being. Inhaled medication delivers drugs directly to the lungs, providing a rapid onset of action and reducing systemic side effects compared to oral or injectable medications. Three primary types of inhalation devices are used to administer these medications: nebulizers, metered-dose inhalers...
761
Higher Mental Functions of the Brain: Language
3.5K
Language is a system of communication that allows the expression of thoughts, ideas, and feelings. The brain processes language in both hemispheres.
Language formation and comprehension take place in the dominant hemisphere. The dominant hemisphere is responsible for understanding the meaning of spoken, written, or sign language, as well as the ability to communicate. For most people, the left hemisphere is the dominant one. The right hemisphere, then, gives tone and emotional context to the...
Language formation and comprehension take place in the dominant hemisphere. The dominant hemisphere is responsible for understanding the meaning of spoken, written, or sign language, as well as the ability to communicate. For most people, the left hemisphere is the dominant one. The right hemisphere, then, gives tone and emotional context to the...
3.5K

