评估推理 大型语言模型与人类类型的思维在眼科问题回答回答
Zhouqian Wang1, Chenjia Xu1, Lei Wang2
1Ningbo Key Laboratory of Medical Research on Blinding Eye Diseases, Ningbo Eye Institute, Ningbo Eye Hospital, Wenzhou Medical University, Ningbo, China.
BMJ open ophthalmology
|January 27, 2026
概括
推理大型语言模型 (LLM) 在眼科问答方面表现得更好,DeepSeek-R1表现出色. 这些模型更好地模拟了人类的思维,为更可靠的眼睛护理人工智能铺平了道路.
科学领域:
- 人工智能在医学中的应用
- 眼科医生 眼科 眼科
- 自然语言处理自然语言处理.
背景情况:
- 在眼科等专业医疗领域评估大语言模型 (LLM) 的功能至关重要.
- 评估LLM的推理过程,而不仅仅是准确性,对于值得信赖的AI应用程序至关重要.
- 当前的LLM可能无法完全复制复杂的医疗问题所需的细微思维.
研究的目的:
- 评估推理大语言模型 (LLM) 在回答眼科相关问题的表现.
- 使用一种新的评估框架,比较推理的LLM与传统的非推理的LLM.
- 分析眼科问答LLM的准确性和推理质量.
主要方法:
- 评估了两个推理法学士 (DeepSeek-R1,QwQ-32B) 和一个非推理法学士 (LLaMA-3.3-70B-Instruct).
- 利用了MedQA-Eye,这是一个定制的数据集,包含10个子专业和3种语言的967个眼科问题.
- 开发了一个新的框架来评估LLM思维模式,模拟人类的医学推理.
主要成果:
- DeepSeek-R1获得了最高的答案准确率 (90.59%),超过了LLaMA-3.3-70B-Instruct (87.90%) 和QwQ-32B (84.28%).
- 错误的逻辑推理是推理LLM的主要失败模式 (93.41%-94.74%的错误).
- 与QwQ-32B (4.31±40.70) 相比,DeepSeek-R1的语义不确定性 (1.04±3.63) 显著降低,这表明推理更可靠.
结论:
- 与非推理模型相比,推理LLM,特别是DeepSeek-R1,在眼科问题回答方面表现优越.
- 这些发现表明,LLM的推理可以更好地模仿医疗环境中类似人类的思维过程.
- 这一进步有助于在眼科中开发更可靠,更复杂的AI系统.
相关概念视频
Reason and Intuition
7.5K
The human brain processes information for decision-making using one of two routes: an intuitive system and a rational system (Epstein, 1994; popularized by Kahneman, 2011 as System 1 and System 2, respectively). The intuitive system is quick, impulsive, and operates with minimal effort, relying on emotions or habits to provide cues for what to do next, while the rational system is logical, analytical, deliberate, and methodical. Research in neuropsychology suggests that the...
7.5K
Reasoning
434
Reasoning is the action of thinking about something in a logical, sensible way. It is integral to problem-solving, decision-making, and critical thinking. Reasoning can be inductive or deductive. Reasoning involves transforming information into conclusions, which is essential for problem-solving, decision-making, and critical thinking.
Inductive reasoning involves deriving generalizations from specific observations. This type of reasoning helps form beliefs about the world. For example,...
Inductive reasoning involves deriving generalizations from specific observations. This type of reasoning helps form beliefs about the world. For example,...
434
Deductive Reasoning
68.0K
Deductive reasoning, or deduction, is the type of logic used in hypothesis-based science. In deductive reasoning, the pattern of thinking moves in the opposite direction as compared to inductive reasoning, which means that it uses a general principle or law to predict specific results. From those general principles, a scientist can deduce and predict the specific results that would be valid as long as the general principles are valid.
For example, a researcher can deduce specific predictions...
For example, a researcher can deduce specific predictions...
68.0K
Counterfactual Thinking
238
Counterfactual thinking is a cognitive process wherein individuals mentally reconstruct alternative versions of past events, often beginning with “what if” or “if only.” This reflective mechanism plays a significant role in shaping emotional experiences and guiding future behavior. Though typically triggered by unfavorable or unexpected outcomes, counterfactual thinking can also emerge in mundane, everyday decisions and experiences, revealing its deep entrenchment in...
238
Language
906
Language is a unique communication system that uses words and systematic rules to organize and transmit information. Unlike other forms of communication, which may involve postures, movements, odors, or vocalizations, language relies on symbols and grammar. This makes human communication distinct from that of other species, who also communicate but do not use language in the same way humans do.
Corballis and Suddendorf (2007) and Tomasello and Rakoczy (2003) highlight the role of language in...
Corballis and Suddendorf (2007) and Tomasello and Rakoczy (2003) highlight the role of language in...
906
Inductive Reasoning
67.3K
Inductive reasoning is a form of logical thinking that uses related observations to arrive at a general conclusion. It is uncertain and operates in degrees to which the conclusions are credible. As such, inductive arguments can be weak or strong, rather than valid or invalid, and conclusions can be used to formulate testable, falsifiable hypotheses.
Inductive reasoning is common in descriptive science. A life scientist makes observations and records them. This data can be qualitative or...
Inductive reasoning is common in descriptive science. A life scientist makes observations and records them. This data can be qualitative or...
67.3K


