大型语言模型的比较准确性,稳定性和校正性在耳鼻喉学和药物监测中
Filippo Bruno1, Lise Sogalow1, Bertrand Blankert2
1Department of Surgery, Research Institute for Language Science and Technology, University of Mons, Mons, Belgium.
与Gemini-1.5-Pro.相比,ChatGPT-4o和Claude-3.5-Sonnet在耳鼻喉科门诊护理中表现出优越的临床和药监性能. 尽管输出稳定,但反并没有提高诊断准确度,这表明这些大型语言模型 (LLM) 的短期可纠正性有限.
科学领域:
- 人工智能在医学中的应用
- 耳鼻喉科 耳鼻喉科 耳鼻喉科
- 药物监督 药物监督 药物监督
背景情况:
- 大型语言模型 (LLM) 越来越多地被用于临床应用.
- 评估专业门诊护理的LLM绩效对于安全的整合至关重要.
研究的目的:
- 为了比较三个LLM (ChatGPT-4o,Gemini-1.5-Pro,Claude-3.5-Sonnet) 的临床和药监性能,稳定性和可纠正性,在耳鼻喉科门诊环境中进行.
主要方法:
- 一个潜在的案例系列涉及51名成年患者在多个大学医院.
- 临床医生处理了患者病史和体检结果;临床准确性和药物警性被盲目耳鼻喉科医生和药剂师评价.
- 在反后重新查询模型以评估可纠正性,并分析了interrater可靠性和稳定性.
主要成果:
- 与Gemini-1.5-Pro.4o相比,Claude-3.5-Sonnet和ChatGPT-4o的临床和药监测性能表现优越.
- 所有的LLM都比临床医生推更多的检查;反并没有显著提高重新查询的诊断准确性.
- 在LLMs中观察到高interrater可靠性 (医生:ICC=0.853,药剂师:ICC=0.991) 和临床稳定性 (Cronbach的α=0.831-0.856).
结论:
- 在耳鼻喉科门诊护理中,ChatGPT-4o和Claude-3.5-Sonnet比Gemini-1.5-Pro.提供了比Gemini-1.5-Pro.更好的临床和药监性能.
- LLM的输出稳定,但在反后的短期纠正能力有限.
- 需要进一步的研究来提高临床决策支持中LLM的准确性和可靠性.
更多相关视频
06:22Machine Learning-Based Cough Tone Classification: Diagnostic Exploration of Chronic Obstructive Pulmonary Disease and Respiratory Tract Infections
Published on: September 19, 2025
04:04Asthma Detection Research Based on Voice Signal Processing and Machine Learning
Published on: July 22, 2025
相关概念视频
Improving Translational Accuracy
Improving Translational Accuracy
Pharmacokinetic Models: Comparison and Selection Criterion
Physiological models take a detailed approach by considering specific molecular processes. They can predict drug distribution, metabolism, and elimination changes, providing a comprehensive understanding of how drugs interact with the body.
Pharmacovigilance
This process, termed pharmacovigilance, aims to detect, evaluate, and minimize harmful effects related to medication use. The data collection for pharmacovigilance depends on spontaneous reporting systems, where healthcare professionals or patients voluntarily report suspected ADRs.
In some cases, there...
Analysis of Population Pharmacokinetic Data
Pharmacokinetic Models: Overview
There are three primary types of models: empirical, compartment, and physiological. Empirical models, with minimal...
