量化LLM在临床病例上的推理能力
Pengcheng Qiu1,2, Chaoyi Wu1,2, Shuyu Liu1
1Shanghai Jiao Tong University, Shanghai, China.
Nature communications
|November 6, 2025
概括
大型语言模型 (LLM) 在医学上表现有前途,但需要更好的推理评估. MedR-Bench揭示了当前的AI在诊断方面表现出色,但在治疗规划和推方面扎,错过了关键的推理步骤.
科学领域:
- 人工智能在医学中的应用
- 临床决策支持系统 临床决策支持系统
- 自然语言处理自然语言处理.
背景情况:
- 大型语言模型 (LLM) 展示了先进的推理能力,但它们在医学中的临床应用和推理评估尚未成熟.
- 在复杂的医疗场景中评估LLM的推理过程对于安全有效的部署至关重要.
研究的目的:
- 引入MedR-Bench,这是一个新的基准,用于评估LLMs的医学推理.
- 评估临床护理不同阶段的最先进的推理LLM的性能.
主要方法:
- 开发了MedR-Bench,包括1453个结构化的患者病例,在13个身体系统和10个专业领域进行参考推理.
- 创建了一个评估框架,涵盖检查建议,诊断决策和治疗计划.
- 使用推理评估器,一个自动化工具来评估推理的效率,事实准确性和完整性.
主要成果:
- 目前的LLM在具有足够数据的简单诊断任务中达到超过85%的准确性,但在检查建议和治疗计划方面却失败.
- 法学士的推理通常是事实准确的,但经常忽略了关键步骤.
- 开源LLM正在缩小与专有模型的性能差距.
结论:
- 虽然LLM在医学诊断中显示出潜力,但在更高层次的临床推理任务中需要显著改进.
- 开发像MedR-Bench这样的强大基准对于推进医疗保健中的AI至关重要.
- 开源技术的进步表明,未来的临床人工智能工具将更容易获得,更公平.
更多相关视频
05:47Evidence-based Knowledge Synthesis and Hypothesis Validation: Navigating Biomedical Knowledge Bases via Explainable AI and Agentic Systems
Published on: June 13, 2025
1.3K
06:48Lexical Decision Task for Studying Written Word Recognition in Adults with and without Dementia or Mild Cognitive Impairment
Published on: June 25, 2019
9.7K
相关概念视频
Language and Cognition
700
Language serves as a bridge between ideas and communication, influencing how individuals perceive and interact with the world. Psychologists have long debated whether language shapes thought or vice versa. This discussion gained grip with Edward Sapir and Benjamin Lee Whorf in the 1940s, who proposed that language determines thought, a concept known as linguistic determinism. They suggested that the vocabulary and structure of a language influence how its speakers think and perceive reality.
700
Higher Mental Functions of the Brain: Language
3.4K
Language is a system of communication that allows the expression of thoughts, ideas, and feelings. The brain processes language in both hemispheres.
Language formation and comprehension take place in the dominant hemisphere. The dominant hemisphere is responsible for understanding the meaning of spoken, written, or sign language, as well as the ability to communicate. For most people, the left hemisphere is the dominant one. The right hemisphere, then, gives tone and emotional context to the...
Language formation and comprehension take place in the dominant hemisphere. The dominant hemisphere is responsible for understanding the meaning of spoken, written, or sign language, as well as the ability to communicate. For most people, the left hemisphere is the dominant one. The right hemisphere, then, gives tone and emotional context to the...
3.4K
Reasoning
388
Reasoning is the action of thinking about something in a logical, sensible way. It is integral to problem-solving, decision-making, and critical thinking. Reasoning can be inductive or deductive. Reasoning involves transforming information into conclusions, which is essential for problem-solving, decision-making, and critical thinking.
Inductive reasoning involves deriving generalizations from specific observations. This type of reasoning helps form beliefs about the world. For example,...
Inductive reasoning involves deriving generalizations from specific observations. This type of reasoning helps form beliefs about the world. For example,...
388
Inductive Reasoning
64.5K
Inductive reasoning is a form of logical thinking that uses related observations to arrive at a general conclusion. It is uncertain and operates in degrees to which the conclusions are credible. As such, inductive arguments can be weak or strong, rather than valid or invalid, and conclusions can be used to formulate testable, falsifiable hypotheses.
Inductive reasoning is common in descriptive science. A life scientist makes observations and records them. This data can be qualitative or...
Inductive reasoning is common in descriptive science. A life scientist makes observations and records them. This data can be qualitative or...
64.5K
Deductive Reasoning
63.7K
Deductive reasoning, or deduction, is the type of logic used in hypothesis-based science. In deductive reasoning, the pattern of thinking moves in the opposite direction as compared to inductive reasoning, which means that it uses a general principle or law to predict specific results. From those general principles, a scientist can deduce and predict the specific results that would be valid as long as the general principles are valid.
For example, a researcher can deduce specific predictions...
For example, a researcher can deduce specific predictions...
63.7K
Critical Thinking II
4.0K
Critical thinking is a cognitive process with several attributes. The attributes of critical thinking include the following:
4.0K
