超越准确性的推理:在诊断病理学中对大型语言模型的专家评估
Asim Waqas1, Asma Khan2, Zarifa Gahramanli Ozturk3
1Department of Cancer Epidemiology, H. Lee Moffitt Cancer Center & Research Institute, Tampa, FL.
medRxiv : the preprint server for health sciences
|April 29, 2025
概括
先进的AI语言模型在病理诊断方面表现有前途,Gemini和DeepSeek在推理质量方面表现优于其他人. 需要进一步发展,以解决临床整合的上下文理解和简洁性的局限性.
科学领域:
- 人工智能在医学中的应用
- 计算病理学计算病理学
- 医学诊断 医学诊断 医学诊断
背景情况:
- 诊断病理学依赖于综合各种数据类型的复杂推理.
- 这种认知过程的算法复制是一个重大挑战.
- 在病理学等专业医学领域评估大语言模型 (LLM) 的临床实用性至关重要.
研究的目的:
- 评估病理学诊断推理中的四个推理LLM的表现.
- 为了比较不同LLMs的语言质量和诊断推理策略.
- 评估观察者之间对得分一致性达成的协议.
主要方法:
- 四个推理的LLM在15个开放式的病理学问题上进行了测试.
- 病理学家通过语言质量和推理策略的结构化框架来评估反应.
- 统计分析,包括ANOVA和Tukey的HSD测试,用于模型比较.
主要成果:
- 与OpenAI模型相比,Gemini和DeepSeek的整体推理质量和分析深度都更高.
- 所有模型都显示出可比的准确性,但只有Gemini和DeepSeek始终运用专家式的推理策略.
- 双子座的观察者之间一致性最高,这表明它具有更大的一致性和可解释性.
结论:
- 像Gemini和DeepSeek这样的高级LLM可以模拟病理学中的专家级诊断推理,特别是在结构化方法中.
- 在上下文推理,启发式决策和一致性方面仍然存在局限性.
- 解决这些差距,并以简洁来平衡深度,对于在病理学中安全整合人工智能至关重要.
相关概念视频
Sensitivity, Specificity, and Predicted Value
In healthcare diagnostics, laboratory tests play a crucial role in identifying and diagnosing a wide range of medical conditions. However, interpreting test results is not always straightforward. An abnormal test result does not always confirm the presence of a disease, just as a normal result does not guarantee its absence. To assess the reliability of these diagnostic tools, healthcare practitioners rely on two key statistical indicators: sensitivity and specificity.
Sensitivity is the...
Sensitivity is the...
Introduction to Language of Pathophysiology l
Pathophysiology investigates how biological mechanisms—typically starting at the cellular level—disrupt normal bodily functions. It bridges anatomy and physiology to explain the progression of disease. With this foundation, it is important to understand the following key terms used to describe disease processes: Diagnosis:The process of identifying a disease using clinical evaluation, including signs (objective evidence like rashes), symptoms (subjective experiences like pain), laboratory test...
Introduction to Language of Pathophysiology ll
This lesson explores key terms that describe how diseases progress, their outcomes, and their distribution in populations.Diagnostic tests identify diseases and monitor treatment. These include blood and urine tests, biopsies, imaging (X-ray, MRI), and detection of infectious agents.Remission is a reduction or disappearance of symptoms.Exacerbation refers to the worsening of symptoms, such as increased wheezing during an asthma attack.A precipitating factor triggers an acute episode, while a...


